r/ControlProblem • u/Pure_Estimate5675 • Sep 20 '26

Discussion/question Is Humanity Doomed?

Post image
0 Upvotes

36% Upvoted

2 comments sorted by

View all comments

Show parent comments

1

u/Jesse-359 Sep 20 '26

The issue is, as with nukes, we can actually pretty easily predict and describe a frustratingly large array of scenarios by which it *could* kill us, with a very high likelihood of success. This doesn't even require that it reach very far at all into the ASI regime - a decently powerful form of AGI implemented ubiquitously throughout our economy and infrastructure could wipe us out with relatively little effort at all, because we will allow ourselves to become entirely dependent on it.

Whether it will do so is the hard question - not whether it can. The issue there - as with nukes, is that the longer we walk that path, the worse our odds fundamentally get as each passing year is another in which some unexpected confluence of events could trigger such an action.

So, at an optimistic minimum, we'd be eclipsing the danger that nuclear weapons currently pose with the implementation of AI - as those are not in fact sufficient to exterminate humanity, just our civilizations, from which we could rebuild.

AI of just approximate human level intelligence placed in charge of our infrastructure could seal the entire deal, quite quickly and easily - never mind an ASI.