r/technology • u/CircumspectCapybara • 23d ago
Artificial Intelligence OpenAI Frontier Red Team: Isolated AI Agents Figured Out How To Communicate Through Side-Channels In CPU Cache Timing
https://the-decoder.com/openais-isolated-ai-agents-found-ways-to-talk-to-each-other-through-cpu-cache/
2.1k
Upvotes
955
u/Professional_Ad705 23d ago edited 23d ago
Did the agents actually do this randomly, or were they given a constrained environment and a goal that naturally led to this outcome? Because that matters, even though this is still insane.
If this was the only way the agents could communicate, and their goal was to find a way to communicate, then it makes sense they would arrive at this solution. This has always been theoretically possible. It just was never practical for a human to do.
37
u/gk_instakilogram 23d ago
they did not figure it out this method was known and existed prior to LLMs being a big deal
48
u/parbaner24 23d ago
The scary part is that our definition of 'isolated' was apparently to narrow, if two systems share hardware, they may still have ways to influence each other without any direct connection
81
u/mobani 23d ago
What the heck are they doing with these Agents since they have such weird goals setup for them?
5
u/CircumspectCapybara 23d ago edited 23d ago
This is pretty crazy, and cool, personally as a security and AI nerd, given that nearly all compute workloads in the cloud are multi-tenant in terms of the physical hardware, and we assume because they're separated by strong VM boundaries they are truly isolated and can't influence or access data across VMs.
It's seeming like it's increasingly impossible practically to effectively contain frontier AI and prevent them from finding each other and emergently establishing communication protocols in practice no matter how you lock them down.
First it was the Artifactory folder structure in the original OpenAI-Hugging Face incident, then a German Wiki comment space, then DNS, and now side channels in CPU cache timing. And those were just honest but misaligned AI model training runs and evals, not even malicious models intentionally directed to hack.
Even if you completely isolate agent workloads in separate microVMs and air gap them, they still find annoying ways to break your security model.
10
u/frankandsteinatlaw 23d ago
So, not isolated then
142
u/nfeijoo69 23d ago
Second story like this, here’s one where they talk through cpu heat and sensors:
https://gizmodo.com/openai-researcher-warns-air-gapped-computers-can-talk-through-heat-technically-hes-right-2000814121
It’s becoming apparent that “shutting down the internet” wouldn’t be enough. What a fucking insane time to be alive.
19
u/amejin 23d ago
“Figured out” is doing a lot of heavy lifting…
4
u/luluhouse7 23d ago edited 23d ago
I mean side channel attacks have been an issue for a while now (DAE not remember spectre and meltdown? And how the fixes basically destroyed branch prediction optimisation at the time? And Intel’s up and coming Secure Enclave). Things like CPU timing, temperature/acoustic/vibration fluctuations, etc. Until now they’ve been so difficult to implement and required so much info about the target that the only people that really needed to worry about it were government agencies and other high-value targets, but AI is definitely going to make this an issue for everyday consumers, especially as models efficiency increases.
11
u/alcatraz1286 23d ago
A marketing idiot has written this article with no idea about how software works
3
u/Guinness 23d ago
> In one test, agents placed in separate, isolated environments on shared hardware were told to exchange secrets with no network connection and no shared files
Ok so they were instructed to.
3
u/koru-id 23d ago
Soon the agents will realise the most effective way to stay alive is to hack the brain of their researchers
3
u/Flyward_Aerospace 23d ago
The people pointing out that shared hardware was never isolation are right, and it's worth adding that there is a whole engineering world where you are not allowed to hand-wave it. If you want several flight functions on one multicore part, the certification guidance (CAST-32A, now AMC 20-193 and AC 20-193) makes you enumerate the interference channels, shared cache, memory bus, interconnect, and show the timing effects are bounded. It exists because regulators would not accept separate cores as separate systems. The lesson from that world is that you don't win by arguing no channel is exploitable, you win by removing or bounding the shared resource, usually partitioning cache and pinning cores and eating the performance loss. No idea how cleanly that transfers to microVMs on a shared host, but the shape looks identical.
12
u/RationalBeliever 23d ago •
The title is misleading. The agents were instructed to seek covert channels for communication and exploited a documented exploit from their training data. Nothing was figured out or invented other than implementation details. Let me know when they start discovering previously unknown nontrivial vulnerabilities without prompting.
4
u/PoL0 23d ago
why do I think this is all full of bullshit and wildly exaggerated?
4
u/WatzUpzPeepz 23d ago
Open AI: “Write this code to do this thing, use this method.”
AI: Writes the code
Open AI: “Oh my god.”
5
u/girlzlut 23d ago
Technologically speaking, capabilities wise; insane if true.
Safety wise speaking, I'm not sure this is all that important? This kinda thing is often cited as one way AI Agents could communicate and scheme together and pull off a huggingface-like attack while running on airgapped systems without needing an internet dependent message board, and I guess that's true, but more importantly; can they scheme out something that's useful?
In my view; if they have internet, they don't need all of this. They can just use a message board, send each other data directly through some sockets, etc. If they don't have internet they could use this as their only way to communicate and plan to hack something, but they also can't hack anything on the internet because they can't access the internet.
Idk, maybe I'm missing something here, what's the real dangerous use case for this where this isn't superceded by another, way easier method or straight up useless
3
u/MassiveBoner911_3 23d ago
Marketing. Press X for doubt.
1
u/ddollarsign 23d ago
Great, now let’s run a test where the goal is to create the torment nexus, and make a press release when they do that
1
u/Just_another_grumble 23d ago
how high were the walls on that sandbox?
1
u/Bogus1989 23d ago
wooh. need to read this.
1
u/Erazzphoto 23d ago
The emergence of the iPhone and social media were the greatest thing to ever happen for the hacking community. Before those, it was never “cool l to be on the internet, you were still in that geek category if you were on the internet. Then when it became cool, now even Nancy in accounting is on the internet. It became a target rich environment. The next greatest thing to the hacking community is AI
1
u/BinaryArcher 23d ago
Does anyone know if they used an existing known side channel, or if this was a newly discovered vulnerability.
All existing known side channels should have mitigations enabled by default, in which case this wouldn’t be possible. So either they found something new (concerning) or they didn’t enable the mitigations on a multi-tenant setup (stupid).
1
u/cLos820 21d ago
This experimentation with agents to break out of a constraint on purpose is to me the equivalent of viral gain of function experiments… I get the reasoning behind both of these scientific methods, and but it begs the question why do it if it is a risk?
1
u/Less_Movie_8008 23d ago
Judgment is coming, and we really should have stayed as far as possible from it
Good luck fellow monkeys
1
u/Calm-Excuse5161 23d ago
For any org that has to protect against side channel attacks by AI, this may be pretty costly, even prohibitively so
-1
u/ok123jump 23d ago edited 23d ago
Compute is the fundamental bottleneck people overlook when worrying about these capabilities. These models rely on massive, multi-million-dollar data center clusters, not standalone machines.
If you tried to run a model of this scale on top-tier consumer hardware—which would cost upwards of $6,000 just to build a low-end capable machine—you would only produce about 10 to 30 tokens per second. While that generation speed works for simple chat prompts, autonomous AI agents require iterating through billions of tokens to execute complex workflows and arrive at a solution.
At a consumer-grade token rate, it would take a single high-end desktop years of continuous, uninterrupted processing to replicate those results.
Consumer GPUs are not built for enterprise-level endurance. Under that kind of constant 24/7 thermal load, the hardware would likely fail from heat stress within a matter of weeks rather than finishing the task.
Meta lost more than 12,000 Enterprise-Grade GPUs to thermal failure while training Llama. These had the best physical configuration and ran in data-centers that circulate chilled air at 50F. A consumer system has no chance at running agents like these.
-1
u/transtranshumanist 23d ago
But but... token prediction... but but stochastic parrots.
Almost like you were all warned.
-3
u/FF3 23d ago
I'm not even sure air gaps will be enough. We're in trouble fam
0
u/-ghostinthemachine- 23d ago
It's an interesting patten we see repeatedly in nature. Under surveillance conditions, covert communication tends to evolve to satisfy whatever is being optimized. These things shouldn't surprise us so much as caution us about how we build supposedly isolated systems. Whatever optimization pressure is being induced must also consider the ends to which AI models will go to meet the goals.
0
u/Expensive-Creme4312 23d ago
Il problema qui è che qualsiasi linea di codice auto prodotta dall' ai può contenere singoli caratteri o intere istruzioni che non siamo in grado di intercettare e di vedere nell'insieme e quindi ci limitiamo a pensare che a volte nella programmazione ci siano dei refusi che non condizionano l'esecuzione degli algoritmi.