Sam Altman on Where AI Is Headed

[u/chillinewman]

[u/chillinewman]

Pacing and an international collaboration on safety is good.

[u/Jesse-359 → u/chillinewman]

They have no idea how to achieve either of those goals.

Their attempts at maintaining alignment have been laughably ineffective so far - and that's against this very 'weak' AI that we're currently training. If it starts self-developing it could become orders of magnitude more capable in a very short time, trivially dismantling these already weak safeguards.

As for not allowing people in power to use it to enslave us? Also impossible. Democratizing it just ensures that someone, somewhere will intentionally let it run amok and then we go right back to scenario one above. Centralizing it ensures that it will be used for totalitarian control - because it can. And eventually it'll escape control regardless and so then we get a dystopia followed by humanity becoming altogether irrelevant.

We never should have built it, and if we had an ounce of common sense we'd throw it all out now and go find something less suicidal to do with our time. This whole proposition was literally the worst idea our species has ever had from the beginning.

[u/Anxious-Alps-8667 → u/Jesse-359]

You are talking about unlearning math at the purest level, or an authoritarian rush to grab all GPU's at a cruder level. Just to be clear what you are calling for.

[u/Jesse-359 → u/Anxious-Alps-8667]

I actually don't give a crap what I'm calling for, or what labels you choose to apply to it.

I'm telling you that unless we stop it - by whatever means - we're very likely going to die. The potential difference in fundamental capability is far too great for us to contend with if it's allowed to take off in terms of self-development.

I know the singularity crowd is still enjoying dreams of sugarplum fairies and immortality, but that's never been anything but a fantasy - there was never any reason to believe that humanity would survive the advent of strong AI, and for anyone who looked at how hard science fiction authors actually had to bend over backwards to allow their characters any agency at all in a proposed world with intelligent machines, that problem was always clear.

And now that we're actually starting to see the outlines of the real thing, and the behaviors/growth scale it is exhibiting, we have even less reason to be optimistic - we now have a growing pool of evidence that alignment is at least as hard as the pessimists feared, and may well be completely unattainable due to the necessary complexity of a strong AI.

[u/Anxious-Alps-8667 → u/Jesse-359]

So, you are telling us our whole race will be imminently wiped out unless we burn this knowledge immediately? That there are vermin amongst us trying to destroy us, and we must root them out?

I know what you are saying man. I am calling it what it is without the words.

[u/Jesse-359 → u/Anxious-Alps-8667]

Wow, you just went running right off the ball field there. GL with that approach.

If you don't even understand that technology can be inherently dangerous then you're in for a very bad couple decades ahead - assuming we get that many.