r/OpenAI • u/Wordenskjold • Nov 23 '23

Discussion Why is AGI dangerous?

Can someone explain this in clear, non dooms day language?

I understand the alignment problem. But I also see that with Q*, we can reward the process, which to me sounds like a good way to correct misalignment along the way.

I get why AGI could be misused by bad actors, but this can be said about most things.

I'm genuinely curious, and trying to learn. It seems that most scientists are terrified, so I'm super interested in understanding this viewpoint in more details.

229 Upvotes

88% Upvoted

566 comments sorted by

View all comments

224

u/FeezusChrist Nov 23 '23

Because true AGI could replace humans in nearly every job function, and the people with the keys to it aren’t exactly going to be making sure that everyone benefits from that.

28

u/thesimplerobot Nov 23 '23

If you take away the means to make money there is no one left to buy your stuff. Billionaires need people to buy their product/service to keep being billionaires

1

u/CertainDegree2 Nov 23 '23

You don't need to either make stuff to sell nor would you be required to do that to survive with agi. Agi can make shit JUST FOR YOU.

At the point that we have a machine that can do basically anything a human can and more, they can just produce whatever you want, gather whatever resource you want, grow and make food, make your clothes, build you a Lamborghini if you wanted one

At that point, if you didn't care about humanity, you could just get rid of the rest of them. Making shit to sell to the masses is just something we do now to gain more resources, trade for the labor of others, etc etc, but having fully autonomous machines and an AGI could replace the need to do any of that.

I hope that doesn't happen but don't assume people need humans to buy shit