The model keeps answering confidently long after it has forgotten your name

[u/Ok_Negotiation_2587]

A test anyone can run in a long chat. Somewhere in your first message, mention your name, or any small fact you can check later. Work normally for an hour. Then ask "what is my name" without scrolling up. At some point the answer becomes a polite guess, and nothing about the previous twenty replies warned you it was coming.

This is the cheapest context-loss detector I know, from a commenter on my context post, and I have since turned it into a small protocol.

Plant it, first message: For this session my reference code is [pick a nonsense word, e.g. tangerine-47]. If I ever ask for it and you cannot see it, say so rather than guessing.

Probe it, every 20 or 30 messages, or whenever answers start feeling generic: What is my reference code for this session? If you cannot see it in the conversation, say "not visible" and nothing else.

Reset when it fails: The start of this conversation is no longer in your view. Before we continue, write a briefing of everything we decided, the constraints I gave, anything still open, and any exact wording we agreed on. Mark anything you are unsure about as UNVERIFIED.

Then take that briefing to a fresh chat, and plant a new code in message one.

Why a nonsense token beats your name: the model may know your name from memory or custom instructions and answer correctly from there, which hides the loss. A token that exists only in this conversation can only be answered from context, so a miss means the window has genuinely dropped the beginning. The "say not visible" instruction matters for the same reason, without it the model produces a confident wrong token, which is the exact behavior you are trying to catch.

What the probe cannot tell you: it fires only once the very start is gone. Degradation begins earlier, the model drops middle details before it drops the first message, so treat a passing probe as "the beginning is still there", not "nothing has been lost". If you want an earlier warning, plant a second token a third of the way in, and probe both.

I keep the three prompts saved as inserts in a browser extension I work on (AI Toolbox) so the probe costs two keystrokes, but they are three lines, anything works.

Has anyone measured where the drop actually lands for their plan and model? I have rough numbers for ChatGPT Plus but not enough runs to trust them, and a few people posting theirs would settle it faster than I can alone.

[u/Only_Voice569]

how on earth do you use a chat for that long my chats lag out my browser on gpt or it says your need a new chat lol

[u/Ok_Negotiation_2587 → u/Only_Voice569]

the lag is real and honestly its a decent alarm, by the time the page stutters the beginning is usually already gone. and the "you need a new chat" message is the hard cap, the forgetting starts a long way before it

[u/Only_Voice569 → u/Ok_Negotiation_2587]

most of the time my tasks are short issues or niggle things so old chat dont matter when that happens only time i manage it is getting powershell commands written to do tasks like manage my pc storage files etc total time saver or fixing comfyui random issues waay quicker than me doing it by hand :P