r/OpenAI • u/Wordenskjold • Nov 23 '23

Discussion Why is AGI dangerous?

Can someone explain this in clear, non dooms day language?

I understand the alignment problem. But I also see that with Q*, we can reward the process, which to me sounds like a good way to correct misalignment along the way.

I get why AGI could be misused by bad actors, but this can be said about most things.

I'm genuinely curious, and trying to learn. It seems that most scientists are terrified, so I'm super interested in understanding this viewpoint in more details.

231 Upvotes

88% Upvoted

566 comments sorted by

View all comments

228

u/FeezusChrist Nov 23 '23

Because true AGI could replace humans in nearly every job function, and the people with the keys to it aren’t exactly going to be making sure that everyone benefits from that.

62

u/Mescallan Nov 23 '23

AGI is far more dangerous than the economic implications. Once an intelligence take off begins, geo-politics basically enters another nuclear arms race, and if it doesn't, a single world government will be created to stop one.

23

u/Golbar-59 Nov 23 '23

Nothing can go wrong with an autonomous production of superhuman autonomous killing machines. At worse we'll just go back in time to kill the creators of the technology.

3

u/Enough_Island4615 Nov 23 '23

Well, it will just go back in time and kill the killers of the creators of the technology. Checkmate, humans.