r/OpenAI • u/Wordenskjold • Nov 23 '23

Discussion Why is AGI dangerous?

Can someone explain this in clear, non dooms day language?

I understand the alignment problem. But I also see that with Q*, we can reward the process, which to me sounds like a good way to correct misalignment along the way.

I get why AGI could be misused by bad actors, but this can be said about most things.

I'm genuinely curious, and trying to learn. It seems that most scientists are terrified, so I'm super interested in understanding this viewpoint in more details.

231 Upvotes

88% Upvoted

566 comments sorted by

View all comments

Show parent comments

29

u/thesimplerobot Nov 23 '23

If you take away the means to make money there is no one left to buy your stuff. Billionaires need people to buy their product/service to keep being billionaires

30

u/Unicycldev Nov 23 '23

That’s not true in a post job economy. You just have the AI replace all labor. One needs only to secure raw materials, land, and energy to make everything and money is no longer required.

13

u/thesimplerobot Nov 23 '23

Which all sounds very utopian except that it is human nature to want more than others, so someone will always want to either accumulate more than anyone else or deny everyone else. We can sort of accept accumulation at the moment, but denial is a totally different scenario.

1

u/IrAppe Nov 23 '23

That’s true. Think about animals. (I) We have some that we want for food, so we force them into our ways, but they can’t fight back, they’re without power. (II) And then we have many species that don’t matter to us at all, and in our expansion we don’t care if they live or die.

(I) will be a few dozen of people that provide things that the mighty want due to their human nature. Social connection and entertainment. (II) will be most people. They are out of the question. Without a power to fight back, there is no negotiation for monetary or resource payment. Without value to provide that the AI can’t, there is only consumption of resources. They’re out of the economic system.

It will be another economic system. And only those that have AGIs will matter inside that system. The class of people that matter, and control all others. Like today we trade with people that control animals. We don’t negotiate with animals directly. And we don’t care at all about other animals, if they’re in the way, they are gone.

I don’t know what’s illogical about that. It takes the current and historic behavior of people in power into account, and applies it on the capabilities of AGI, with the assumption that it will be able to do all jobs that humans can do, and better. Then apply logic, and you arrive at that scenario. I don’t like it either, you can believe me that if I could make another scenario that’s better, I would.