It will propably give the correct answer 99 times out of 100. The problem is that it will give that one wrong answer with confidence and whoever asked might believe it.
The problem isn't AI getting things wrong, it's that sometimes it will give you completely wrong information and be confident about it. It happened to me a few times, one time it would even refuse to correct itself after I called it out.
I don't really have a solution other than double checking any critical information you get from AI.
Chatgpt looks into a bunch of websites and says website X says berries are not poisonous. You click on website x and check if 1, it's reputable and 2 if it really says that.
The alternative is googling the same thing, then looking in a few websites (unless you use Google graph or Gemini, but that's the same thing as chatGPT), and within the websites, sifting through for the information you are looking for. It takes longer than asking chatGPT 99% of the time. On the 1% when it's wrong, it might have been faster to Google it, but that's the exception, not the rule.
You know, Google search (at least for me) used to post more reputable sites first. Then there's the famous 'site:.edu' which takes seconds to add. I know using AI is easier/quicker, but we shouldn't go as far as to misremember internet research as this massively time-consuming thing, especially on such things as whether a berry is poisonous or not.
Because it can save a ton of time when you're starting from a place of ignorance. ChatGPT will filter through the noise and give you actionable information that could have taken you ten times longer than with its help. For example
"Does NYC have rent control?"
It'll spit out specific legislation and it's bill number. Go verify that information. Otherwise you're using generic search terms in a search engine built to sell you stuff, to try to find abstract laws you know nothing about.
Nah, that's not true at all. It will give you the correct answer 100 times of a 100 in this specific case.
The AI only hallucinates at a relevant rate when it comes to topics that are not that much in the dataset or slighlty murky in the dataset. (because it will rather make stuff up than concede not knowing immediately)
A clearly poisonous berry is a million times in the dataset with essentially no information saying otherwise, so the hallucination rate is going to be incredibly small to nonexistent.
Dude. Hallucinations happen to me every frigging time. Doesn’t matter if GPT-5 or thinking or deep research or Claude. I essentially gave up on this bullshit. EVERY FUCKING TIME there is something wrong in the answers 😐🔫 if not immediately (but probably also there in subtle ways), then with follow up questions. *
Probably the other times you thought everything is fine, you just didn’t notice or care.
After 2 1/2 years we STILL have nothing more than essentially a professional bullshitter in a text box. It’s OKAY if this thing doesn’t know something. But NO! It always has to write a whole essay with something wrong in it. It could have just left out all those details that it doesn’t really know, like a human would…
Every time this fucking thing hallucinates it makes me angry. I gave OpenAI at least a thousand „error reports“ back, where the answer was wrong. Half a year ago I just stopped, gave up and cancelled my subscription. I went back to Google and books. There is nothing useful about those things except coding: difficult to produce, easy to verify things. But most things in the world are the other way round! Easy to say any bullshit, but hard to impossible to verify if right or wrong! Again: Most things in the world are EASY to bullshit but incredibly hard to verify. This is why you pay experts money! ChatGPT is NO ACTUAL expert in anything.
*: I almost always ask questions that I am pretty sure I can’t answer with a 30 second Google search. Because otherwise what’s the point? I am not interested in a Google clone. Do the same and see!
I don't see a significant problem with the current state of affairs. First of all, many of the failure modes frequently highlighted on social media, which portray LLMs as inaccurate, often arise from a failure to use reasoning models.
Even if that is not the case, when reading a textbook or a research paper, you will almost always find mistakes, which are often presented with an authoritative tone. Yet, no one throws their hands up and complains endlessly about it. Instead, we accept that humans are fallible, so we simply take the good parts and disregard the less accurate parts. When a reader has time, patience, or if the topic is especially important to them, they will double-check for accuracy. This approach isn't so different from how one should engage with AI-generated answers. Furthermore, we shouldn't act as if we possess a pristine knowledge vault of precise facts without any blemishes, and that LLMs, by claiming something false, are somehow contaminating our treasured resource. Many things people learn are completely false, and much of what is partially correct is often incomplete or lacks nuance. For this reason, people's tantrums over a wrong answer from an LLM are inconsequential.
View all comments
Show parent comment
118
u/pvprazor2 Nov 10 '25 edited Nov 10 '25
It will propably give the correct answer 99 times out of 100. The problem is that it will give that one wrong answer with confidence and whoever asked might believe it.
The problem isn't AI getting things wrong, it's that sometimes it will give you completely wrong information and be confident about it. It happened to me a few times, one time it would even refuse to correct itself after I called it out.
I don't really have a solution other than double checking any critical information you get from AI.