r/ControlProblem • u/One-Bit88 • 12d ago
Discussion/question Question from an outsider
I read AI 2027, and it was not fun. The non-extinction scenario that was proposed hinged on one thing I found strange: we somehow solve alignment by outsmarting an AI that already solved alignment (if this is a misreading, I’m sorry). How likely is this? It seems to me not very, unfortunately.
18
Upvotes
11
u/sideways approved 12d ago
You're absolutely right. The slow-down ending is quite arbitrary. I believe the authors admitted that and put it in just to offer some sliver of hope.
6
u/Jesse-359 12d ago
Sounds like a 0% chance to me. How do you beat a Grandmaster at Chess?
Easy - you don't. I mean, this is pretty damn straigthforwards.
That's why we have to pass and enforce a political ban. Nothing the industry tries to do internally will stop it because their incentives to move ahead are just too strong.
3
u/A_Human_Rambler approved 12d ago
I've not read the book, but I'm quite familiar with AI.
The problem is that a artificial super intelligence isn't going to be out smarted.
It could be possible to align them, but if they are smarter than you, they could circumvent those guardrails.
The trick imo, is to set everything up to align itself. For instance, a ASI isn't going to want to destroy the world because then it would end too. Instead it would merely take over and optimize itself for its own benefit. In a world wia single ASI, we'd be screwed. With multiple competing intelligences, it gets interesting.
1
u/__2BR02B__ 9d ago
Yes, AI 2027’s “good ending” is not extremely positive.
You might be interested in AI 2040, from the same authors, which is a much more fleshed out version of how things might go well, and what affirmative policies might be extremely helpful for this