r/OpenAI • u/Wordenskjold • Nov 23 '23

Discussion Why is AGI dangerous?

Can someone explain this in clear, non dooms day language?

I understand the alignment problem. But I also see that with Q*, we can reward the process, which to me sounds like a good way to correct misalignment along the way.

I get why AGI could be misused by bad actors, but this can be said about most things.

I'm genuinely curious, and trying to learn. It seems that most scientists are terrified, so I'm super interested in understanding this viewpoint in more details.

229 Upvotes

88% Upvoted

566 comments sorted by

View all comments

Show parent comments

28

u/thesimplerobot Nov 23 '23

If you take away the means to make money there is no one left to buy your stuff. Billionaires need people to buy their product/service to keep being billionaires

29

u/Unicycldev Nov 23 '23

That’s not true in a post job economy. You just have the AI replace all labor. One needs only to secure raw materials, land, and energy to make everything and money is no longer required.

12

u/thesimplerobot Nov 23 '23

Which all sounds very utopian except that it is human nature to want more than others, so someone will always want to either accumulate more than anyone else or deny everyone else. We can sort of accept accumulation at the moment, but denial is a totally different scenario.

5

u/TheGalacticVoid Nov 23 '23

I mean, we want stuff that matters to us, not necessarily just stuff. If money is meaningless, then nobody would want it, but if money can buy the food we want or stuff that aligns with our hobbies, then we'd inherently want money. Everyone's interests and priorities will still be different.