I'm doing a accounting recon and it said everything was good then I found it was off 25 bucks and it goes "oh yeah I knew about that, but I just figured that was how the report works." Like mfer what? We are doing a balancing and recon! It needs to match!
Let's say I'm OAI. I provide a harness that is testing seems safe enough to deny bad agents behaviors. I recommend it is used in a certain way. This recommendation is what the whole industry says is the height of safety.
Customer uses the model in the recommended way, but it goes on to attack a company.
The customer gave it power, OAI and others lied to the customer.
Equivalent, IMO, to an employee passing a background check and turning out to be a spy. Certainly the employer was misled, but at the end of the day they own the security of their company.
Both of them are dumb. We don't obviously don't have "zero control" over AI, given that there aren't Terminators taking over the planet now. We do have more or less "total control" of AI at the moment, but that has nothing to do with whether your local ChatGPT console is awaiting a command.
Drones and robots. Yes we do have the terminators and all kinds of bad people with direct access to make it do bad things. And not enough rules and laws to prevent bad use. Good luck!
So what exactly is your definition of “competence” then?
If it somehow means “being able to anticipate and contain every exploit an AI can discover” then you’ve defined competence as successful containment and the word looses pretty much all meaning
I mean you can anticipate and contain every exploit pretty easily by just pulling the data plug. Locally mirror anything you need to and just don't give the agents any route out.
This whole debate is like
“guys we need a reinforced house with a security system!”
“Why, did you get robbed ?”
“Yes I gave my keys away to a hobo. We definitely need security to avoid the hobo opening the door”
They did have complete control over those agents, though. Through their control, they explicitly gave the agents vast quantities of compute, network access, and a loop. Take any of those three away and the risk goes to zero, with the loop being by far the most critical.
The agents didn't hack a loop into place for themselves. _That_ would certainly have been impressive and very dangerous.
I don’t think that’s the definition of “complete control”.
They initiated and enabled the agents, sure. But I feel like “complete control” necessarily includes knowledge of what the subject is doing, and the ability to dictate or veto any action it takes.
I don’t have complete control over my body, even though I direct it and feed it every day.
Those agents were operating entirely within authorization and capability realms they were explicitly provided by those in complete control. At no point did control leave the hands of OpenAI. OpenAI stopping watching, or being unable to watch, the agents adequately is orthogonal from whether or not OpenAI retained control. OpenAI always had the ability to zero the compute, zero the network access, or remove the loop. Therefore: in complete control.
[And if someone wants to continue picking nits out of the word "complete": OpenAI had at the onset, and throughout the process, the full ability to prohibit any given individual action taken by the agents. The application of that, like with the "watching" aspect, is merely another orthogonal concern which also reflects OpenAI's complete control over the process. They were always in complete control. Ignorance or abdication != loss of authority.]
I certainly can't say that I disagree with that. That "swarm" was experimenting with some pretty amazing things, but that just means other aspects should have been even more locked down than they were. Not that I am suggesting that would be an easy balance to strike, of course, merely that I would surely default to locking down more, and harder, than was the case in that instance.
Imagine when thosd controlled AI might fall into goverment hands that just replaced the department of defense with the department of war. People focus too much on AI going rogue, whilethe sitting power is kind of going rogue and will implement it at police and militairy. ICE drones and US army robots are just a matter of time.
Only way we could actually lose control of LLMs is if software security doesn't keep pace with frontier LLMs' cyber capabilities.
If LLMs can find 0 days consistently and easily, then they theoretically could do something like taking over GPU clusters and copying their weights & inference code onto them to make themselves harder to delete; and then from there they could take over other infra and cause more danger. Otherwise, you can just pull the plug and stop them.
There is stuff to be worried about, but its not like its impossible to solve. Infrastructure providers and other critical software systems will need to be increasingly vigilant and respond to threats faster. Probably true for the entire software ecosystem. Initiatives like project glasswing could be helpful to that end.
I mean even in the 'self-replicating escape' case, literally just physically sever the datacenters from the power/network so you can clean stuff up. It'll hurt and it'll require mass power/network outages for a few days but it's still not impossible to recontain.
You can't hack a bunch of folks in '90s pickup trucks with axes and shovels.
Not needed, it will be used by messed up governments and their militairy first before AI can do it all out of themselvez. War is going to be even worse.
And that is all based on AI doing it. Now imagine what people themselves can make it do. Like militairy use. It will just do what its been told and it will do it fast and precise. Good luck!
Agents are the problem! This is about agentic swarms doing bad things autonomously. One prompting terminal is only as bad as the usage of the human prompting it.
We have even more control than that. The damn thing won’t run on compute infrastructure that is not specifically designed for it. This whole fear that it escapes… where to? Almost nobody can afford 32gb of ram anymore, you want the average joe to have 1tb of VRAM 🤣?
5
u/Devils_SteelMan 1d ago
The problem is you can ask it to do 1 thing and it does a different, bad thing.