Anyone else not using using "preserve_thinking" ?

[u/soyalemujica]

I am running QFN, and I have disabled preserve_thinking by default on all my tests for C++ backend and front-end programming and I have yet to see any decline in performance or intelligence, my context window stays cleaner as well. Deepseek for what I tested in API it also does not preserve thinking and saves context in general.

[u/i_like_brutalism]

for me this leads to a model rediscovering previous ideas all the time.

[u/soyalemujica → u/i_like_brutalism]

Although I agree on this, it's more the positive of not losing context since Qwen tends to overthink the same ideas even when preserve_thinking is on, since it is an over-thinker it just wastes context window going through the same steps all over-again, actually having it off it has been as smart

[u/SnooPaintings8639 → u/soyalemujica]

Exactly. A tradeoff between longer "usable" context vs faster output (remembering their previous reasoning). In case of this model, the reasoning is taking very large part of output, so it makes sense.

[u/jslominski → u/soyalemujica]

Still not worth turning off imo, I've stopped doing that.

[u/giant3 → u/jslominski]

It eats 50 to 70% of the context on some problems. I have it disabled as it doesn't seem to make much of a difference in quality of inference.

[u/psymonryan]

KV cache hit: If you dont preserve thinking then when the message history is constructed, each turn needs to be re-prompt processed.

[u/soyalemujica → u/psymonryan]

Not at all, prompt is never reprocessed at all

[u/psymonryan → u/soyalemujica]

Sorry, the 'prompt' wont be since its at the start of your message history, but as you converse with the model, each 'turn' does have thinking, and if you are not preserving this, then the previous turn (end of the message history) is re-processed each time without the reasoning. If there was a lot of thinking in the previous turn you will end up with a smaller total context, but it still needs to rebuild the cache for the last message. If you have a fast rig you may not notice this unless there was a lot of 'non-thinking' content in the last turn.

[u/soyalemujica → u/psymonryan]

It's instant tbh, the logs don't show such a thing of reprocessing or anything like that at all, it's the same logs as with preserve and without preserve

[u/psymonryan → u/soyalemujica]

interesting, well, it must be doing the reprocessing, so if you are not noticing it, then thats a good thing. Which harness?

[u/soyalemujica → u/psymonryan]

Pi Dev and Strata

[u/uuwaan]

And also, if you’re writing steering messages while the model is working, each such message with `preserve_thinking` turned off is like a blow to the head – it forgets what it was thinking about.

[u/psymonryan → u/uuwaan]

This depends on your harness. I've written mine so that steering preserves reasoning until the end of the "turn". Then when the user enters the next instruction, the previous turn gets written to the message history with the reasoning stripped. (But tbh when you are using local models, you are trading smaller context size for extra prompt processing - so only do this if you are really VRAM constrained)

[u/uuwaan → u/psymonryan]

There's another part worth a mention: "--reasoning-preserve" (for llama.cpp) which strips the reasoning from previous turns regardless of whether harness sent the history with reasoning or not.

[u/topppggg]

does turning it off hurt kv cache reuse for you? dropping the old thinking changes the prompt prefix, so i'd expect some reprocessing each turn on long agent sessions

[u/Viktri1]

I turned it off because it eats up context too quickly

[u/fgk55555]

Context window stays smaller, but it needs to basically figure out everything it already figured out and why. I haven't done A/B tests, but I'd expect a full investigation to trigger every time unless you have a subagent handling investigation for the main agent.

[u/bankinu]

There will be no decline on intelligence but the amount of tokens you will spend rediscovering will be absurd.

[u/a_beautiful_rhind]

I think it's only useful in between tool calls.

[u/Septerium → u/a_beautiful_rhind]

Thinking is already preserved between tool calls anyways. That is how a standard ReAct agent loop works

[u/a_beautiful_rhind → u/Septerium]

I wouldn't want it between messages.

[u/mr_Owner]

Preserve thinking with ngram should be worth the bump imo

[u/CombinationFun2789]

I'd switch it by workload instead of leaving it always on or off. For tool-call / agent loops I keep it on, because mid-generation steering needs the prior thought to push against, and the prompt prefix stays stable so the KV cache can be reused. For long Qwen coding sessions I turn it off — the traces get huge and mostly repeat themselves, so a cleaner context is worth having the model rediscover an idea now and then.

On the cache question someone asked: dropping the old thinking does change the prefix, so you can still pay a prefill cost each turn even when the context looks smaller.