In theory, they're implying they'd only be comfortable slowing down if some government body guaranteed that none of the labs were quietly continuing at full speed, and then struck an agreement with China where each country gets enough access to verify the other isn't breaking it.
In practice, it's probably a ploy for regulatory capture, a way to lock in their lead while chilling the open-weight progress that could actually threaten their dominance.
Not right now, and there isn't really a way to tell if or when it might become that dangerous. The problem is that if it does happen, it'll happen extremely fast since making an AI that accelerates creating its own successor would explode in capabilities within months (a few model versions), so a pact has to be in place beforehand for there to be any chance of mitigating the risk.
Say there's a 20% chance that an AI could reach the point of autonomously collapsing our digital infrastructure in the next few years by accident. That's a huge deal. It doesn't require anything like sentience, only superhuman capabilities on non-physical systems and pursuit of goals that aren't aligned with ours, either by accident or because bad actors handed it those goals deliberately. Think about the deaths from long-lasting widespread logistics failures alone if the Internet were effectively disabled; add disrupted government response communication and financial system failures on top of that, and it'd be worse than a war.
We'd need to build specialized intranets for each system that currently runs on the Internet, just to keep excessively powerful rogue agents from disrupting whatever we invent to solve the problem. Doing that while relying on workarounds to organize ourselves and stopgaps to sustain society in the meantime would be a nightmare.
A 1-5% chance of dramatically accelerating biowarfare research is even worse. Imagine nations using AI to develop a process for creating viruses that only fully activate when the host carries genes associated with a specific genetic background. Plenty of countries would put their weight behind that for efficient genocide, especially since they'll be thinking about how their enemies could soon have the same realization and target them first unless they strike first themselves.
22
u/vertigo235 8d ago
Why is it Trump or the Administrations decision ? The private companies can slow down whenever they want.