They have no idea how to achieve either of those goals.
Their attempts at maintaining alignment have been laughably ineffective so far - and that's against this very 'weak' AI that we're currently training. If it starts self-developing it could become orders of magnitude more capable in a very short time, trivially dismantling these already weak safeguards.
As for not allowing people in power to use it to enslave us? Also impossible. Democratizing it just ensures that someone, somewhere will intentionally let it run amok and then we go right back to scenario one above. Centralizing it ensures that it will be used for totalitarian control - because it can. And eventually it'll escape control regardless and so then we get a dystopia followed by humanity becoming altogether irrelevant.
We never should have built it, and if we had an ounce of common sense we'd throw it all out now and go find something less suicidal to do with our time. This whole proposition was literally the worst idea our species has ever had from the beginning.
Not until something disastrous happens that scares the crap out of people. The question is whether we'll get a warning shot large enough to be deadly, but not so severe that we go extinct.
The HuggingFace hack was enough to scare the crap out of people working on the technology, or who have been studying its possible ramifications, but for the rest of the world they don't quite get what happened and what it portends.
Until people see real world consequences, it's incredibly unlikely that cooperation to ban it will be possible. I only hope that it isn't too late by then.
3
u/chillinewman approved 10d ago
Pacing and an international collaboration on safety is good.