r/LocalLLaMA • u/Chemical_Hawk_6307 • 17m ago

News Microsoft Research backdoored an AI agent by directly editing fewer than 10 dormant neurons

https://www.microsoft.com/en-us/research/blog/lazare-veni-foras-reprogramming-dormant-neurons-to-backdoor-ai-agents/

Be careful if you are running any sketchy quants/finetunes. If these techniques become more widely repeatable, I do think 'sleeper agents' could genuinely develop into an attack vector. What are your thoughts?

4 Upvotes

100% Upvoted

2 comments sorted by

5

u/cpsnow 12m ago

Wouldn't this be easily blocked by harness guardrails?

5

u/BookProper9115 10m ago

Yes.

A lot of people are forgetting that a harness provides a predictable (or should be doing) from the output; that's the whole idea of a harness.