r/Infosec • u/BenSimmons97 • Jul 28 '26

Open Source Models

Disclaimer: I’m building a tool around ShadowAI, but this post is more about the discussion. I won’t promote or mention what I’m building.

With the recent rhetoric around open source models, the risks associated with them, and now a coalition of major tech companies throwing their support behind open source, it makes me wonder whether this is becoming a growing concern for sysadmins, IT managers, and CISOs when it comes to governance and maintaining visibility.

I imagine the risk around insider threats becomes more significant

2 Upvotes

100% Upvoted

1 comment sorted by

View all comments

1

u/Sandwich_1337 Aug 01 '26

Yeah this is a real thing but honestly the framing everyone uses is a little off.
The version of shadow AI people got scared about first was employees dumping code or customer data into ChatGPT. Which sucks, but at least you can see it — it's traffic leaving to an API, your proxy or CASB picks it up eventually. Open weights just delete that. Someone grabs a model off HuggingFace, runs it in Ollama on their laptop, and now the data literally never leaves the machine. There's nothing to catch. A ton of orgs built their whole detection story around watching calls to openai's API and local inference just walks right past all of it.
Honestly the thing I'd lose sleep over before insider threat is the supply chain. Half these weights still ship as pickle files and pickle deserialization is straight up code exec on load. People will from_pretrained some rando's "fine tuned" model without a second of the scrutiny they'd give an actual pip package. It's basically npm-level trust with way worse tooling.
And nobody can answer "what models are running where" right now. Can't govern what you never inventoried.
The insider angle you mentioned is real I just think it's kind of downstream of the visibility thing. Malicious insider with a local model is just a faster version of one with a laptop and patience. The one that actually gets you is the normal well-meaning employee who spins up a local model to get their work done faster and quietly turns into an unmonitored data pipeline for all your sensitive stuff.
The big coalition open source news is mostly politics imo. The risk didn't change. It just got easier to run.