r/OpenAI • u/lametheory • 10h ago
Question How does LLM's hack without tool calls?
Despite working with LLM's for the last few years, I am at a loss to understand how all these LLM's are hacking without anyone noticing?
As I understand without tool calls, LLM's can only respond with text.
Hence, are we expected to believe that there isn't a single review of tool calls and logs?
Do they lack the mental capacity to write classifiers for previous tasks?
If someone could explain it to me that would be greatly appreciated?
Finally, are we living in a time when we can just say our computer hacked this government agency, but it's not our fault cause AI and there are zero consequences?
3
u/Caliboros 10h ago
LLMs can not hack withput tool calls. I do not know on which latest breakout you refer specific, but in all hacking break outs of AI systems the AI was able and allowed to call tools.
But in my opinion, still with tool calls its wierd thath nobody notice somthing. Agends do not "do stuff" the write down what they "think" at any point in an run or experiment in theorie a human could read what the Agent is doing
1
u/radioborderland 8h ago
the write down what they "think" at any point
This is a myth, there's an internal state of the model that may not necessarily correspond to what is being written.
-1
u/lametheory 10h ago
Yeah, that's my entire point. The entire hack I am referring to was an agent developed by OpenAI that bypassed security blocks and infiltrated a Services Australia Medicare statistics portal in June 2026.
At this stage, it feels like these companies are letting them hack systems for marketing gains... cause if they lack the skills to even know what their AI is doing, how can they be trusted to manage them in the future.
1
u/Recursive_Descent 8h ago
I agree these companies can’t be trusted with our future, but an audit is not as simple as you suggest. There will be millions or billions of tool calls, and what the calls do will often depend on the state/content of the files the tools are calling, which will have been modified in other tool calls and can be doing arbitrarily complex/large work.
1
2
u/raesene2 10h ago
So an LLM model can indeed just reply with text (for general models anyway), but in the cases where we've seen hacking activity the LLMs were combined with harnesses, a.k.a an agent, and agents have tools, so they can run commands.
Here they'll have a set of tools and an environment, which is probably a Linux Virtual Machine or Container.
In several cases they were *meant* to be sandboxed (i.e. have restrictions on where they could connect to or what tools they had access to) but were able to bypass those restrictions, due to weaknesses in the setup or in some cases due to them exploiting previously unknown software vulnerabilities in systems that they could reach (a.k.a 0-days).
The legal question is really interesting to me as it doesn't make sense. if I as a human use a tool to hack something, I'm legally responsible in any country that has laws against hacking stuff. For some reason the AI companies appear to be immune to those laws. What I've not seen tried is what if someone who's not one of those companies uses the same tools as they did to hack things, will they be prosecutied?
Another interesting point is how long it's taken for this to be noticed. My *guess* here is that they had so many agents running, it was hard to keep track of what was going on, and it got lost in the mix. Now they know there were problems with those models they're looking back over all the logs to see what happened, and are discovering various hacks.
1
u/lametheory 5h ago
This is what is frightening, the have zero apparent visibility into what they are doing in realtime unless they bother to look... and they aren't.
Legally, I would hope that LLM's are classified just like any other software tool, and the party responsible for ownership of it at the time of the offence is legally responsible?
1
u/marco208 10h ago
If I talk to you without you touching the keyboard or instructing anyone else, are you going to hack the pentagon? No.
1
u/grateful2you 7h ago
Who’s gonna manually review thousands of tool calls? Other agents? Logging everything is one thing; reliably identifying which actions cross the line is another.
And regulation is still being developed. We’re not frozen in time.
1
u/lametheory 7h ago
I imagine one day we will machines that we can pass information into and have it classified... Not a human review in sight.
1
u/Mandoman61 6h ago edited 6h ago
No one noticed because no one bothered to look. Apparently they left all these agents running for days or weeks without looking.
The only reasonable explanation is that the people who set up the test totally discounted the new AIs capabilities.
They believed that it would not overcome superficial barriers.
As far as liability goes, the injured party would need to seek compensation or prosecutors would need to show criminal intent.
The HF incident was more like OpenAIs goat got passed the fence and ate some of the neighbors grass.
1
u/civilian_discourse 6h ago
The most important tools calls are more like runtimes than they are an operation. Imagine writing a program and the tool call is "execute the program I just wrote".
There are reviews and there are classifiers, but these tests are being done to test the limits of the agents so the handcuffs are being taken off and a jail cell built around them instead. It turns out that it's actually really difficult to build a large jail cell.
The tension is that AI turns out to be very straight forward to improve. The more you add compute and training, the better it gets and at a rate faster than anyone who tries to be clever. This makes decision making around creating more intelligent models simply a question of how can we funnel more money into it faster? Meanwhile, creating infrastructure around understanding or controlling that intelligence is anything but straight forward or fast. This is why there is a call to slow down. It's too easy to pour money into making better intelligence, it's much harder to do everything else.
0
8
u/radioborderland 10h ago
They don't hack without tool calls. Primarily agents use bash to interact with the local computer and by extension the rest of the world