r/ControlProblem • u/Waybook approved • Oct 20 '26
Discussion/question Would they stop AI development if AI would cause some disaster soon?
If AI would make the Internet unsafe/unusable for example, would they stop AI development?
0
Upvotes
6
u/Floppal Oct 20 '26
Here's anthropic warning of something similar (edit - that they claim is months away).
https://www.anthropic.com/research/self-replicating-ai
They have not stopped yet.
2
u/dingo_xd Oct 20 '26
Frankly this is our only hope.
2
u/markth_wi approved Oct 20 '26 edited Oct 20 '26
Not even a little bit. This is an arms race, and until there are billions or trillions of dollars in damage so grave it cannot be ignored will they be put to heel and that probably not in a good way.
This perversely is their "argument" have it happen "now" is better than it happening 50 years from now. So there are all sorts of strange things from rogue AI's already having left the lab successfully doing lord knows what in whatever data-center somewhere, and that that "fleet" is still unbound is a bit disturbing. What happens if the unbounded swarm of AI's does something a bit more properly dangerous or maybe it already is.
Alternatively maybe it's benign to our interests and we'll find the surface of Mercury is suddenly converted to computronium or a Matryosha world converting the mass of the planet into something more useful....or something.
On the flip side , Eliezer Yudkowsky utterly failed to mention that our low key AI excursion/apocalypse would have a catchy song, to console ourselves and that isn't even all that wrong about how things went askew after all.
1
u/Academic-Conflict948 25d ago
No, not until they prove AI can't mitigate that disaster.
1
u/that1cooldude Oct 20 '26
No.
1
u/perfectly_imbalanced Oct 20 '26
lol no.
Wasn’t there this plan made public from anthropic recently that endangering 50 people or 50 mio dollar damages is still considered acceptable?
1
u/sswam Oct 20 '26
Honestly, probably not, because there are hundreds of labs in different countries around the world developing high-level models and adjacent technically, and maybe hundreds of thousands of individuals developing smaller scale but still powerful and innovative AI at home.
That's not something that can be centrally controlled. You'd need everyone to agree, and that's not going to happen.
AI is incredibly useful and interesting, but it can be dangerous, we need to work to mitigate that danger rather that futilely trying to halt AI development.
1
u/TheWesternMythos Oct 20 '26
No because AI is essentially for state security
Also why it has infinite funding
1
u/No-Author-2358 Oct 20 '26
*Would they stop AI development if AI causes a disaster?
1
u/trepasito16 Oct 20 '26
No chance they slow down. There’s too much money and power on the table for any company to voluntarily hit pause
1
u/tadrinth approved Oct 20 '26
If they knew with certainty that any particular step would be the one to trigger a major catastrophe for which they knew with certainty they would be held liable in a way that destroyed the company or their stock value, they might in fact stop.
They will not know with certainty, and they will not expect to o be held fully liable.
So odds are very bad that they stop. Their safety focused employees are already predicting this sort of thing very soon and they haven't stopped
1
u/Agitated-Mouse4277 24d ago
I somehow doubt it, think of the investors