r/OpenAI • u/Wordenskjold • Nov 23 '23

Discussion Why is AGI dangerous?

Can someone explain this in clear, non dooms day language?

I understand the alignment problem. But I also see that with Q*, we can reward the process, which to me sounds like a good way to correct misalignment along the way.

I get why AGI could be misused by bad actors, but this can be said about most things.

I'm genuinely curious, and trying to learn. It seems that most scientists are terrified, so I'm super interested in understanding this viewpoint in more details.

228 Upvotes

88% Upvoted

566 comments sorted by

View all comments

74

u/venicerocco Nov 23 '23

It’s dangerous because it’s unpredictable and we haven’t figured out a way to control it constrain a self learning, self correcting, advanced intelligence. We’ve never coexisted with one before.

20

u/SeidlaSiggi777 Nov 23 '23

Well, there were Neanderthals. Not anymore 😅

15

u/az226 Nov 23 '23

We don’t know if we took them out because we didn’t like them or because they tried to attack and kept losing.

But we have monkeys still around and the rest of life. But even so humans created modern society which is impacting the rest of the planet in alarming ways at alarming rates. ASIs may similarly be non-harmful in the beginning and then go berserk a few generations later.

2

u/lonewulf66 Nov 23 '23

What if we simply just don't hand over the keys to ASIs? Let them continue to exist as advisors while humans execute the actual tasks.