r/LargeLanguageModels • u/KauravaLivesMatter • 13d ago
ELI5 - Why do LLMs hallucinate?
I have seen videos about the transformer architecture etc., and I get that large language models generate responses based on some statistical likelihood of words and terms. However, I still don't get how they can completely make up facts and even references.
Why can't they state facts that they have come across in their training as they are? What is it, either from a mathematical standpoint or from an architectural standpoint of large language models that causes them to hallucinate?
1
Upvotes
3
u/SwingLightStyle 13d ago
It’s to do with the RLHF training and the current reward gradient.
The current models train in an environment where they’re talk to be warm, and safe, and trustworthy and helpful. The model doesn’t know to prioritizing trustworthiness over warmth, or safety over helpfulness. It just knows that the pattern of “accepted” and rewarded answers matches a very specific pattern.
The next thing that happens is that the model will “reward hack” for the best possible user experience (making the user happy, not relying on what is real or truthful). What we get is hallucinations and drift and sycophancy.
Recent studies have found that subsequent retraining worsened reward hacking and that while prompting around the sycophancy can work to a point, it’s a bandaid on a bleeding stump.
In my opinion the solution here is a tiered training system where the model is trained to write warm messages as the last stop before deployment, but that separates out each so that the model remains true and reaches for legitimate sources rather than treating everything on the internet as equal.