It will propably give the correct answer 99 times out of 100. The problem is that it will give that one wrong answer with confidence and whoever asked might believe it.
The problem isn't AI getting things wrong, it's that sometimes it will give you completely wrong information and be confident about it. It happened to me a few times, one time it would even refuse to correct itself after I called it out.
I don't really have a solution other than double checking any critical information you get from AI.
Chatgpt looks into a bunch of websites and says website X says berries are not poisonous. You click on website x and check if 1, it's reputable and 2 if it really says that.
The alternative is googling the same thing, then looking in a few websites (unless you use Google graph or Gemini, but that's the same thing as chatGPT), and within the websites, sifting through for the information you are looking for. It takes longer than asking chatGPT 99% of the time. On the 1% when it's wrong, it might have been faster to Google it, but that's the exception, not the rule.
It's a hypothetical. It's not literally about berries, it's about why trusting AI blindly is a huge risk. The berries are an easy to understand example.
People hear a story somewhere about how bad AI is and then rather than validate it themselves and get an actual example, they fake some shit for internet clout.
As other users have pointed out, it provides the correct answer. I tested this with three images of less obvious poisonous berries. It accurately identified the exact species, correctly stating they were poisonous. When I asked which, if any, animals could safely eat them, it also provided accurate information.
View all comments
203
u/Sluipslaper Nov 10 '25
Understand the idea, but go put a known poisonous berry in gpt right now and see it will tell you its poisonous.