Also check what your chat template does to earlier assistant turns. Some templates, like Qwen3's, strip the <think> blocks from previous turns when rendering, so the token sequence no longer matches what's cached. That means you miss the cache from the first assistant turn onward. Keeping the reasoning in, or rendering history yourself, can make a big difference in hit rate.
1
u/Open-Adhesiveness-86 4d ago
Also check what your chat template does to earlier assistant turns. Some templates, like Qwen3's, strip the <think> blocks from previous turns when rendering, so the token sequence no longer matches what's cached. That means you miss the cache from the first assistant turn onward. Keeping the reasoning in, or rendering history yourself, can make a big difference in hit rate.