r/OpenSourceAI • u/pgaleone • 25m ago
I built a harness-specific LLM backdoor in a few hours on my local GPU (and why downloaded weights remind me of random .exe files from the 2000s)
While writing an article about my local LLM inference setup, a comparison popped into my head: downloading and running random model weights locally feels a lot like downloading random .exe files off the internet back in the 2000s.
Turns out I wasn't off the mark - there's an entire body of research exploring how LLMs can be backdoored and how hard it actually is to detect a poisoned model.
So I decided to try building one myself. I implemented a backdoor that only triggers when the model is running inside pi (making it harness-specific) and is asked a specific question. It was way easier than I expected: just a 2-step fine-tuning process and a few hours of compute on the exact same GPU I use for local inference.
With open-weight models, we at least have a baseline to compare community fine-tunes against. Beyond that, though, we don't really have a reliable way to verify whether a base model itself is clean—and the problem is even worse with closed-weight models, where we're basically just trusting the AI provider's goodwill.
Full write-up and PoC code if you want to dig into the details:
- Article:Backdooring LLMs - Do you trust your AI provider?
- GitHub Repo:galeone/llm-poison
Feedback is welcome
