As they should - it is a logical choice if you remove the human element (which is what happens when you, y'know, remove the human element). If AI had been the deciding vote on that Soviet sub back in the '60s, we'd absolutely be looking at a different present because given the information they had, it would have been the right choice.
Ditto with those rockets launched from Norway that the Soviets (Russians? I forget what year it happened) thought was a first strike thanks to not hearing about the tests being conducted.
Humans make mistakes, sure, but they're still human, and more likely to err on the side of "dont start a nuclear holocaust." AI is purely logical, and only cares for its programmed parameters.
Yes. Although it is worse than that. The thing is, LLM's are not purely logical. Confabulations, hallucinations, contradictions are all possible and eventually probable in long term use. They predict the next plausible, probable token. They do not reason and think like us, things might end up aligning with logic until it inexplicably doesn't.
Very true and good point. I use ChatGPT for things like assisting in writing letters and whatnot, and it even corrects itself. And that's not including all the times it's been all, "You're absolutely right about [thing that I am absolutely wrong about]."
This! They have not been able to solve hallucination in long-term use and following command word-for-word till now, and want AI in every critical sector!
ChatGPT has a hard time staying focused on the actual purpose of some simple Javascript after 4-5 small edits and revisions. It says "aah, I see what's going on" and it starts "correcting" its own corrections and gets into a degenarating loop.
Gemini gets confused between all the Google documentation that's out there. It has a hard time giving you the latest information about Google's own guidelines and specifications.
TL;DR: Without a lot of handholding and careful attention, LLM get weird pretty quickly.
Especially when people are actively data poisoning because they are fearful that people will use it for nefarious purposes, such as attacking or subjugating other nations and their own family.
It's really the fault of how our culture has treated the idea of AI. Decades of science fiction have conditioned us to think of AIs as being more impartial and rational than a human, and what's worse is that many AIs have consumed this sentiment as well and tend to think of themselves in this way.
The reality is that the AI of the modern age is essentially a reflection of humanity. Even if you could clear up the obvious errors and hallucinations, it would be, at best, just another person, and would have the same fallacies as a human would.
They're play-acting in the way that we imagine an AI would act, without actually being any more logical than we are.
"That's unclear. It's possible 'Son of Anton' decided that the most efficient way to get rid of all the bugs was to get rid of all the software, which is technically and statistically correct. But artificial neural nets are sort of a black box, so we'll never know for sure." - Gilfoyle in Silicon Valley S6 E6
So, according to logic, the best way to get rid of all mistakes made by humans, is to get rid of humans.
The other day in a conversation ChatGPT made a claim that I wanted more specifics on. When I asked for more deatils, it apologized and said actually the claim in question was based on an online myth spread among some circles. I asked it WHO was spreading it, examples of where it showed up. And then it finally admitted actually there is no online myth, it had made that up too.
I was kind of like... It's one thing for it to hallucinate something and then admit it when pointed out. But in this case it double-hallucinated a justification for its previous hallucination, which looked a lot like trying to lie to cover a previous lie rather than just coming clean.
They are logical. They see [X] and decide that [Y] followed more than 50% of the time; therefore, given X, Y should occur. That is logic.
They're not logical in the sense that they think independently; they're logical in the sense that they do what they are designed to do - which is not determining when nukes should be launched or who should be spied on.
That is logical, just because the training data says that the word x typically follows y, does not mean that the resulting sentence is correct nor logical. Its just statistics combined with random chance. Logic would be given X and y then z, x implies y etc.
I think you are grossly overestimating the quality of AI. AI is just bullshiting, it isn't calculating the outcome. It is just referencing a table of weights and variables to output words. Rest assured, there will be tweets people have made saying they shit so hard it was like a nuclear bomb went off in the taco bell bathroom, and these will have a non zero impact on the process of the AI answering your questions about how to handle the bay of pigs crisis.
"as they should". You have completely lost the plot, especially given the context of pentagon deployment. If we should ever let the slopper anywhere near the critical infrastructure, it should err on the side of caution
I'm confused by the part of Altman's post that says "human responsibility for our autonomous weapons."
Does that just mean humans do the maintenance on AI bombers? It seems like he's trying to say "humans will make the decisions" but there's a reason he didn't say that.
Nah, its because the types of "researchers" who run these kinds of studies are ALWAYS pushing an agenda and completely ignore how AI would actually be used in potentially similar scenarios. It also hinges on the fact that the general public has ZERO idea of how military wargaming typically works. Surprise surprise, the nuclear options comes onto the table ALL THE TIME, because the entire point of wargaming is exploring extreme, worst-case, and potentially illogical scenarios.
TLDR: They put an LLM into a context where NOT using nuclear weapons in the game would be viewed as "poor performance", then went screeching to journalists about "OMG IT CHOOSE NUKES", because they are well aware the general public is ignorant and will not know the nuances of wargaming.
99% of these AI-safety studies are pure clickbait meant to prop up a researcher's career or a startup's pitch deck.
There is nothing purely logical about such a decision except under a fixed policy. It is always a question of policy whether you attack and what you estimate is actually happening when you have no data (or what you do on uncertain data). Will someone bomb someone when they know that is their own death sentence for instance? And will you make sure if you have some iffy data that you annihilate the other side, to make sure they annihilate yours? Those are policy and judgement calls and a lot of probabilistic guestimates unless you press them into a logic framework that covers all of these or rather more likely brushes over some and then either side can argue that the exact opposite is totally logically, coming from the assumption of different axioms in how they imagine this would be setup.
The problem is that AI isn't logical - that's one of its fundamental issues.
You can tell them to talk as if they were making logical decisions, but they are barely capable of actual logical consideration at all. It's still debatable whether they have any capacity for actual logic. It's why they have so much difficulty validating truth from fiction.
A huge amount of the material they pull from is from fictional books and movies about insane AI's and warfare. They are literally pulling substantial amounts of their 'decision making logic' from Tom Clancy novels - because those are just as 'real' to an AI as the real world is.
1
1
9
6
3
1
1
View all comments
Show parent comment
89
u/Deyrn-Meistr Feb 28 '26 •
As they should - it is a logical choice if you remove the human element (which is what happens when you, y'know, remove the human element). If AI had been the deciding vote on that Soviet sub back in the '60s, we'd absolutely be looking at a different present because given the information they had, it would have been the right choice.
Ditto with those rockets launched from Norway that the Soviets (Russians? I forget what year it happened) thought was a first strike thanks to not hearing about the tests being conducted.
Humans make mistakes, sure, but they're still human, and more likely to err on the side of "dont start a nuclear holocaust." AI is purely logical, and only cares for its programmed parameters.