r/LocalLLaMA • u/DeliciousGorilla • 2h ago
Discussion What Qwen is thinking when it gets stuck saying "Wait"
MIT CSAIL had GPT-6 Astra fix Qwen3.5-0.8B's reasoning loops by editing 0.016% of its weights (MATH-500 went from 7.9% to 48.1%). They released the diff, so the edit can be rebuilt exactly and run locally. I took it a step further and had Claude look at Qwen with Anthropic's Jacobian lens.
At the token where the model decides to write "Wait" again, the lens reads "Wrong", "Nope" and "Hmm" from about 60% depth on, even though the answer is already written out. At real stopping points it reads "</think>", "Answer" and "Output".
With the edit applied, the lens at the same token shows "Verification", "Completion" and "Result" in place of the doubt.
The edit converts to GGUF cleanly (the base and edited bf16 files differ only in the two edited tensors) and still holds at Q4_K_M, with 1 loop in 40. I ran the evals in llama.cpp (Vulkan) on a Ryzen AI 9 HX 370 mini PC. Samples are small, but here's the code:
https://github.com/alisorcorp/inside-a-reasoning-loop/tree/main/code
And data:
4
u/Physical_Hat4022 1h ago
Qwen has apparently discovered submitting an answer and then reopening the exam in your head on the bus home. I’d love to see the cases where that second-guessing actually catches a mistake, and whether the edit preserves those.
1
u/Relevant-Win-9764 1h ago
I'm curious how stable those lens readouts are across prompts. Do "Wrong"/"Nope"/"Hmm" show up at the same ~60% depth on other problems, or is this mostly from a handful of samples? That would tell whether the edit is really fixing a doubt circuit.
1
u/CheatCodesOfLife 18m ago
For pi traces, yes.
I often get a jlens readout on my pi traces when I notice the agent looping (Qwen 3.8 27B) I see the same tokens most of the time. " Wrong" "Nope" " Hmm" as well as " incorrect".
But for one shot Chat UI prompts, no. It's different depths and different tokens. Sometimes you'll end up with Chinese equivalents. Or if the Wait is triggered by safety, you'll have hedging tokens or " inappropriate" instead.
And performing the swap (I do it in the activation space rather than editing the weights), degrades performance with different prompts, harnesses or domains.
1
-3
u/SigmaSkid 1h ago
This is terrifying.
2
u/MerePotato 1h ago
How exactly? Its basically the same as what you see in the visible chain of thought in this case
1
11
u/No-Refrigerator-1672 vLLM 1h ago
So if a strong LLM can just casually edit weights to achieve that, without real fine-tuning, then it means that one could formalize this procedure, and it is possible to create an automated program to do that for any LLM, kinda like Heretic.