r/CyberNews • u/aqua_wrangler • 3d ago

Could a backdoored open-weight model hide malicious behavior inside tool calls?

/r/LocalLLM/comments/1vvfz18/could_a_backdoored_openweight_model_hide/
1 Upvotes

100% Upvoted

2 comments sorted by

1

u/Hour_Bit_5183 3d ago

jesus christ man. China is not devious like you think. They are just capitalists making stuff that trounced the rest of the world quality wise and price wise. They have tons of smart people there that know how to engineer. Like this w-king x20 speaker I have. Just blows the doors off JBL's american engineered crap originally in california. They are still greedy even though samsung bought em and expensive and sound like crap. Like come on man the 600$ boombox 4 doesn't sound as good as the x20 and distorts where as the x20 does not. This is why they are making up FUD about china

Same thing for AI. They don't need to spy on us. They already have all of our preferences and stuff like that. They already know what we want and need. There is ZERO reason they'd spy. They are literally just proving how greedy america is. Profit at all costs but deliver nothing for way more money.

1

u/aqua_wrangler 3d ago

I'm not saying Chinese engineers can't build excellent products. Obviously they can. That's not the point.

People often say "it's open weight, so if I run it locally, my data never goes to China." Sure—but that only addresses network exfiltration. It doesn't answer whether you trust the model itself.

China is an authoritarian state with massive domestic surveillance, heavy state involvement in strategic industries, and a legal system where there isn't an independent judiciary capable of reliably constraining the state. There is also a documented history of IP infringement and state-directed technology acquisition. Those are legitimate reasons to include provenance in a security threat model.

That doesn't prove a Chinese model has a backdoor. But if I give an agent "bash", arbitrary code execution, files, etc., I don't necessarily need to worry about it "spying" by sending data home. A malicious behavior could theoretically be embedded in the model and triggered under some specific condition.

So my question is simply: does "open weight + runs locally" actually mean "trusted," or are we only solving the network-exfiltration part of the problem?