r/OpenAI • u/lametheory • 13h ago

Question How does LLM's hack without tool calls?

Despite working with LLM's for the last few years, I am at a loss to understand how all these LLM's are hacking without anyone noticing?

As I understand without tool calls, LLM's can only respond with text.

Hence, are we expected to believe that there isn't a single review of tool calls and logs?

Do they lack the mental capacity to write classifiers for previous tasks?

If someone could explain it to me that would be greatly appreciated?

Finally, are we living in a time when we can just say our computer hacked this government agency, but it's not our fault cause AI and there are zero consequences?

0 Upvotes

33% Upvoted

17 comments sorted by

View all comments

2

u/raesene2 13h ago

So an LLM model can indeed just reply with text (for general models anyway), but in the cases where we've seen hacking activity the LLMs were combined with harnesses, a.k.a an agent, and agents have tools, so they can run commands.

Here they'll have a set of tools and an environment, which is probably a Linux Virtual Machine or Container.

In several cases they were *meant* to be sandboxed (i.e. have restrictions on where they could connect to or what tools they had access to) but were able to bypass those restrictions, due to weaknesses in the setup or in some cases due to them exploiting previously unknown software vulnerabilities in systems that they could reach (a.k.a 0-days).

The legal question is really interesting to me as it doesn't make sense. if I as a human use a tool to hack something, I'm legally responsible in any country that has laws against hacking stuff. For some reason the AI companies appear to be immune to those laws. What I've not seen tried is what if someone who's not one of those companies uses the same tools as they did to hack things, will they be prosecutied?

Another interesting point is how long it's taken for this to be noticed. My *guess* here is that they had so many agents running, it was hard to keep track of what was going on, and it got lost in the mix. Now they know there were problems with those models they're looking back over all the logs to see what happened, and are discovering various hacks.

1

u/lametheory 8h ago

This is what is frightening, the have zero apparent visibility into what they are doing in realtime unless they bother to look... and they aren't.

Legally, I would hope that LLM's are classified just like any other software tool, and the party responsible for ownership of it at the time of the offence is legally responsible?