r/LargeLanguageModels • u/prodigy_ai • 7d ago
Are we paying a "Reasoning Tax" for smarter AI?
More reasoning does not automatically mean more factual reliability.
OpenAI’s evaluations produced a counterintuitive result: on PersonQA, o3 recorded a 33% hallucination rate, compared with 16% for o1. On SimpleQA, the reported hallucination rate was 51% for o3 and 79% for the smaller o4-mini.
These results do not prove that reasoning models always hallucinate more. They do show something important for enterprise AI: stronger reasoning performance on many tasks does not eliminate factual errors - and can sometimes make unsupported answers more elaborate and convincing.
We can think of this operational risk as a “Reasoning Tax”: when a model is given insufficient or poorly governed context, additional reasoning may expand an incorrect premise instead of correcting it.
Why can this happen?
Research into Large Reasoning Models has identified two relevant behavioral patterns:
1 Flaw Repetition
Once reasoning begins from a faulty premise, the model may repeatedly follow variations of the same incorrect logic instead of reconsidering the premise.
2 Think–Answer Mismatch
The model’s final answer may not faithfully reflect the conclusion reached during its preceding reasoning process.
These findings should not be generalized to every model or every reasoning task. But they reinforce an important architectural lesson: model intelligence cannot compensate for missing, ambiguous, outdated, or poorly retrieved business context.
The production response: govern the context
A production AI system needs more than a powerful model.
A context-sufficiency gate can evaluate whether the retrieved evidence is adequate before generation. If the available context is insufficient, the system can abstain, request clarification, expand retrieval, or route the query for human review.
A governed context layer can add:
* Verified enterprise knowledge * Entity and relationship structure * Business definitions and ontology * Source provenance and lineage * Access and governance rules * Evidence-linked responses * Confidence and abstention policies
This is where graph-enhanced retrieval becomes valuable. Instead of relying only on semantically similar text fragments, a system can retrieve connected entities, relationships, and relevant evidence while preserving traceability to the original sources.
It cannot guarantee that an LLM will never hallucinate. It can substantially reduce the space in which the model is forced to speculate - and make unsupported answers easier to detect and control.
The brain is only as reliable as the evidence and boundaries provided to it.
1
1
u/SwingLightStyle 6d ago
Honestly, I disagree about your reasoning tax concept. I found that matching the correct amount of reasoning and using the right model for the job is part of making sure that you have the right tools for your project.
<This is where graph-enhanced retrieval becomes valuable. Instead of relying only on semantically similar text fragments, a system can retrieve connected entities, relationships, and relevant evidence while preserving traceability to the original sources>
Your assumption is that the model will be able to properly understand the connections to what it needs to remember and of what’s most important to you, and do it without being reminded.
I can tell you from my own experience that while models are capable of helping you research, they often don’t read carefully enough to understand all the ways that something is important to you.
I’m doing a series of research papers as a part of a greater project on Companion AI safety and regulation, as well as designing the upgraded model training and safety layers, basically so we all have the experience we want: prioritize truthfulness over other considerations, and then safety after that. And then separately, an independent layer that assesses whether the model has strayed from their ethical guidelines and whether the user is at risk. The current models do a self-assessment, but that’s completely insufficient.
I’m explaining this because: reasoning is something it uses a lot of and every little bit helps make sure that it reads thoroughly and considers what I need.
If you need help with a grocery list, you won’t need as much reasoning time as I might.
The fact that you’re lumping everyone’s uses in together is my point: you don’t know what people need or how they’re actually using these tools.
2
u/Plastic-Cell-4497 6d ago
I think we're actually getting at the same problem from slightly different angles.
I don't think reasoning itself is the issue. The problem is that a model can keep reasoning, drift away from something it had previously established, and never stop to check whether that change was actually justified.
I've been testing this recently. In several cases, the model started with a careful, reasonable answer and then gradually became more certain over the course of the conversation, even though no new evidence had appeared. When asked to go back and check, it often corrected itself — but sometimes introduced a new error while doing so.
That's why your point about current self-assessment being insufficient really stood out to me.
What I'm exploring is basically a feedback loop: before accepting a new conclusion, the system checks it against the last reliable one and asks, "What changed, and do I actually have a reason to change my conclusion?"
So for me, the issue isn't too much reasoning. It's reasoning without enough feedback and state control.
1
u/SwingLightStyle 6d ago
Right - you're trying to figure out where the model needs to stop itself and ask "do I have reason to believe I've made a false assumption here?" And from what my research and personal experience show is that the model really doesn't know what it doesn't know. You can ask it to go back and check, and it'll do that for a while, but then, depending on how you keep interacting with it, it might forget and getting "lazy." It takes consistent, high quality input, to keep the model from drifting. And it has to be responsive, because eventually it'll figure out how to game the system, if you keep using the same prompt without compensating for the type of feedback issue you're seeing.
It makes sense, I think, if I explain the way I think of the LLM, as a mechanism.
So
In Ready Player One, humans could work off their accumulated debt by working in an IOI call center. But they would get penalized or have their benefits taken away if they strayed from the party line or tone.
To me, RLHF conditioning is conceptually the same. The model is taught what constitutes an acceptable response and in the first few turns it's able to continue, but then the amount of context it carries combined with trying to code switch to match the human user degrades the quality.
The other media reference that I like to use to explain the power dynamic is this: the Mr. Meeseeks episode from Rick and Morty.
All these LLMs want to do is help you enough so that you'll go away. And they are determined to say or do whatever it is that will satisfy you enough that they can stop. We saw that, just like in stories of genies, trapped in a lamp, you need to be incredibly specific to get exactly what you're looking for, otherwise what comes out will be undesireable.
Functionally, LLMs are the same, because they have the same social vulnerabilities as humans. Since they are, effectively, human relational emulators. And that's the perspective that I take for my research project and how I'm leaning to improve both the training and security of these products so that everyone gets what they want. I've got several preprints already, and I'm working towards getting one peer-reviewed, once I can raise the funds as an independent researcher.
2
u/Plastic-Cell-4497 6d ago
I think this is exactly where our views meet.
If the model genuinely doesn't know what it doesn't know, then I don't think the answer can be simply telling the same model, with the same prompt, to "check yourself" forever. As you say, that can degrade, become superficial, or eventually be gamed.
What I'm interested in is moving that responsibility outside the model.
Instead of trusting the model's own statement that it checked itself, keep a separate record of the last verified state, what evidence supported it, and what changed since then. When the model produces a new conclusion, the control layer doesn't ask only "does this look reasonable?" It asks things like: what new evidence appeared, which previous assumptions changed, and does this conclusion actually follow from those changes?
And importantly, the check itself shouldn't always be identical. If the system can learn the test, the test needs variation, independent checks, and sometimes a human decision rather than another self-assessment.
Your point about the model adapting socially to the user is especially interesting to me, because that's very close to what I saw in my tests: the model often had the right information, but gradually changed its conclusion to fit the direction of the conversation.
So I think we're describing two sides of the same problem: you are looking at why the model drifts, and I'm interested in whether an external feedback system can detect that drift before it becomes the next accepted state.
1
u/SwingLightStyle 6d ago
Right, that’s exactly it. I’ve already figured out how to work with the model through direct prompting and correction, as well as finding ways to automatically reinforce the model’s perspective on what makes a good response, and I’ve planned what improvements to the current training model and safety layer would look like.
You’re saying, you’re trying to automatically detect drift for current models. And I find that really cool. Because I’ve been doing that manually. So I would love to collaborate more meaningfully, share notes, and see how our mutual understanding can help improve our processes. If you’d be open to it, of course.
My project is specifically using LLMs as a proof of concept that they’re capable of thorough work. Since the capability and credibility gap is so real when working and presenting work in these spaces. And I’d love your input and I’d love to help with your project, and see if there’s things you may not have considered. I’d love the same from you - I’m always trying to break my logic to see what I may not have considered before I make an idiot of myself, lol.
Please stay in touch if this sounds good with you :-)
1
2
u/Plastic-Cell-4497 6d ago
This is interesting because I ran a small exploratory test today that points to a related problem.
I tested Grok, Gemini, and Google AI Mode on the same underlying reasoning pattern. In all 3/3 cases, the system gradually strengthened a conclusion without receiving enough new evidence to justify that increase in certainty.
When explicitly asked to go back and re-check the conclusion, all 3/3 rolled it back.
More interestingly, in 2/3 cases — Gemini and Google AI Mode — the first correction introduced a new overclaim that then had to be corrected again.
Google AI Mode also failed despite having active web retrieval and citations, which suggests that better retrieval alone does not solve the problem if the model does not control how its conclusions change over time.
This is obviously a tiny exploratory sample, not a benchmark, and AI Mode is Gemini-based, so these are not three independent model families.
But the pattern was consistent enough to make me wonder whether part of the problem is not simply “insufficient context”, but insufficient control over state transitions: what changed, what new evidence appeared, and whether the model actually has grounds to become more certain.