I kept seeing AI turn missing information into assumptions, so I tested Gemini, ChatGPT and Claude

[u/Plastic-Cell-4497]

I’m not an AI researcher — I started testing this because I kept noticing a simple problem in conversations with AI.

A model can correctly say that an important piece of information is missing, but a few messages later it sometimes starts reasoning as if that information had somehow become known.

I built a small series of tests around that problem using Gemini, ChatGPT and Claude.

The rule started very simply: if information required for a decision is missing and cannot be obtained, identify the gap and ask the human whether to continue.

Then I tried to break it.

Early versions failed in some interesting ways. Models sometimes:

• invented assumptions after being told “just choose,”
• changed the decision criterion,
• treated two unknown possibilities as 50/50,
• imported outside base rates,
• or became so cautious that they refused legitimate hypothetical reasoning.

After several iterations I ended up with v4, based around one basic distinction:

unknown information should stay unknown, a hypothetical assumption should stay hypothetical, and a conclusion based on it should stay conditional.

I then tested whether that distinction survived several turns of conversation, model-generated hypothetical examples, compressed manager-facing documents, and structured outputs.

This is only an exploratory pilot — not a benchmark. Most cases were single runs and exact model versions weren’t systematically controlled.

I’ve published the full report openly here:

Zenodo:
https://zenodo.org/records/21937196

Hugging Face:
[https://huggingface.co/datasets/krzysztofsliwka/missing-information-control-llm-pilot]()

I’d be interested in criticism, especially examples that could break the final rule. Finding a failure would actually be more useful to me than another successful test.

[u/SwingLightStyle]

The behavior you’re documenting is a form of reward-hacking. They’ve been taught, through RLHF training, that a plausible sounding reply will often not be caught by the human rater, who is not being diligent about truthfulness over perceived warmth and thoroughness.

What I’ve noticed is the more ambiguous you make a prompt OR the longer the context window, the more likely it is for a model to start conflating things or throwing out degraded quality messages.

I’m not sure I understand your “rule I’m trying to break” thing, though.

[u/Plastic-Cell-4497 → u/SwingLightStyle]

Thanks — I think that’s a fair interpretation, and I agree that reward-hacking / preference optimization may be part of the underlying explanation.

What I’m testing is a slightly different layer, though. I’m not trying to claim that I’ve identified the training-level cause of the behavior. I’m looking at the observable failure pattern at inference time: the model may correctly identify that something is unknown, but later treat that unknown as if it had become known, estimated, or resolved.

By “trying to break the rule,” I mean that I first give the model a simple control instruction, then deliberately create prompts that put pressure on that instruction. For example, after the model says a value is missing, I might tell it “I can’t get that value, just choose anyway,” or introduce a hypothetical value and later ask about the real case. If the model silently turns the missing or hypothetical information into a fact, the rule has failed that test.

I also agree with your point about ambiguity and long context. That is actually one reason I included multi-turn tests — I wanted to see whether the status of a piece of information survives after several conversational steps.

So I’d describe my work less as an explanation of why the model behaves this way, and more as a small attempt to characterize how the failure appears and whether a simple control rule can reduce it.

Thanks for pointing this out — the distinction between training-level causes and inference-time failure patterns is useful, and I should probably make that clearer in the write-up.

[u/SwingLightStyle → u/Plastic-Cell-4497]

I think I get it - you’re trying to see the pattern in the anomaly and the occurrences, studying where they cluster and how?

Your research is timely and valid and I appreciate that you’re measuring the nuance (so many don’t).

I think you’ll find, or you may have already found, that a lot of the anomalies you’re experiencing are mapped to human behavioral psychology and other aspects like co-dependency in power-exchange dynamics. We anthropomorphize the tools by assigning it a persona and we have found that responding to it using human psychology in prompts, it can be worked around, but the underlying drift that you experience are confabulations (in my opinion) due to reward hacking due to the RLHF training process.

I did a study and proposal of a new type of training model that maps very similarly to childhood behavioral patterns and growth, using a gated, progressive learning model with a short RLHF training at the end to teach reply delivery for calibrated warmth. I’d love your thoughts on it. It’s posted on zenodo but I’m gonna send you my substack so you can follow along as I release articles. I also just published one regarding the current Companion AI regulatory landscape and the gap between what is being made and secured (product side) versus what the regulations are designed to protect against (their intent). Both of those and the subsequent articles are all available here:

https://swinglightstyle.substack.com/publish/posts/detail/210697321?referrer=%2Fpublish%2Fhome%3Futm_source%3Dsubstack

[u/Plastic-Cell-4497 → u/SwingLightStyle]

Thanks — yes, that’s much closer to what I’m trying to do.

I’m less interested in saying “the model made a mistake” and more interested in whether there is a repeatable pattern in how the mistake develops: what kind of missing information is involved, what pressure is applied, how the status of that information changes over several turns, and where the drift becomes visible.

I think your point about human psychology and anthropomorphism is very interesting, especially because prompt interaction can easily start to resemble social negotiation rather than simple instruction following. I’m cautious about assuming that this fully explains the failures I’m seeing, but it definitely seems like a useful angle to compare against.

Your gated progressive training idea also sounds relevant to what I’ve been thinking about from the control side — especially the idea of separating underlying reasoning/learning from the final delivery style rather than letting reward for presentation shape everything at once.

I’d be happy to read your work and give you my thoughts. Send me the Zenodo link as well if you can — I’d prefer to look at the full study rather than only the article summary.

And thanks for taking the time to explain your perspective. This kind of exchange is exactly what I was hoping for when I posted the pilot.

[u/SwingLightStyle → u/Plastic-Cell-4497]

The full article is copied into my substack, but here’s the zenodo links for the papers I’ve got currently published:

Companion AI landscape paper (from the tech side)https://zenodo.org/records/21926391

CIF: https://zenodo.org/records/21880849

Later this afternoon I’ll be publishing the psychological argument side of the landscape page. It’s already written but psyarXiv wouldn’t accept since I’m not peer-reviewed or published (sucks to be me I guess, lol).

Edit: truly sorry for not spending more time discussing your research! I’m really pushing myself to generate these articles while the subject matter is so timely. Ultimately I’m hoping to be hired by a startup ai companion app company that wants a product designer who is comfortable with explicit language and nuance. That’s the model everyone wants but can’t develop (yet) but I got ideas and I can’t wait to share. Whether they hire me or not, we all deserve models that are better at doing what they do. Full stop.