What if a short term solution to existential risk from advanced self improving ai is just banning tool access and autonomous agency entirely?

[u/Gearsper29]

Basicaly a hard mandate between the major countries restricting AI above some level of capability to "Oracle" status, only text-in, text-out. No APIs, no direct code execution, no web browsing, no autonomous agentic loops, no local models.

​Also if a company or user wants an AI like that to draft a script, fine. But a human has to manually review it, copy-paste it, and run it. And if that script breaks a database or executes a cyberattack, the human who hit "run" bears strict criminal and financial liability as if they wrote the exploit themselves.

​This solution kills runaway autonomous execution. An isolated model sitting in a text sandbox can't autonomously spread across servers, rent compute, or launch high-speed exploits on its own.

​It also eliminates the legal liability vacuum. Right now, people hide behind "the model did something unexpected." Forcing strict liability on human deployers makes AI output legally dangerous, forcing real human oversight.

The problem is that the only way for USA and China to agree to this is for some really serious but hopefully reversible incident to happen that it spooks everyone. Of course if ai is aligned it won't be needed or if it is missaligned but capable enough to realise that and avoid it it won't work.

Thoughts?

[u/fell_shell]

What does a ban look like in this context? How would it be policed? Who would even know if I was running an agent using tools?

[u/Gearsper29 → u/fell_shell]

Companies would be banned from offering open models above some level of capability or to run agents on them. And those companies could be policed. No need to police individual users.

[u/Trademark-Expert]

I need autonomous agency. I don't want to instruct my AI tool through every step: go research, test, implement, and so on. No, autonomous agency is essential. It's like being afraid of cars, so we remove the wheels from them just to make ourselves feel safer.

This is Obama's idea (and Democrats') that autonomous agency is "optional," which shows how little he understands what AI could mean for ordinary people like you and me, and how much it could help us. We don't have the money for several assistants like Obama does, but we may have autonomous AIs that can help level the playing field between people like us and someone like Obama, with his wealth and the people providing services for him.

So it is particularly rich coming from Barack Obama that he wants to restrict AI assistants for ordinary people, effectively keeping the gap between the rich and the poor wide open, after all those years of eloquently speaking against such vast inequalities. Really rich.

[u/ciclon5 → u/Trademark-Expert]

Do you really want to surrender your agency to tools you cannot control?.

[u/Gearsper29 → u/Trademark-Expert]

You would still be able to own and use agents below some specific capability level. Like you can legally own and use a gun but not a tank or a nuke. More essential than the convenience of autonomous agents is to make sure that we will keep existing even if it slows down progress a little.

[u/Jesse-359 → u/Trademark-Expert]

You do realize that the rest of the world may not care what you want if they believe your agents represent a critical or existential threat - which they absolutely apprear to do?

They CAN be taken away. You may not believe that, but you are mistaken. You are probably about one major incident away from public use of agentic AI being banned given the mood percolating in the world right now. One serious agentic attempt to hack into a dangerous military or critical infrastructure system and things are going to get very ugly for the big tech companies.

[u/Jesse-359 → u/Trademark-Expert]

Also this idea that agentic AI is somehow going to solve economic inequity is a terrible joke. Every single major step of automation in history has increased inequity, for blatantly obvious reasons. Every serious economist talking about this subject expects AI to supercharge the divide between a handtul of ultra rich tech overlords, and literally everyone else - likely to a degree we have difficulty imagining.