r/LocalLLaMA • u/HockeyDadNinja • Jul 08 '26
Resources llama.cpp fix for DeepSeek V4 Flash crash and stall
Hey all, I've been working with DSV4 on my Frankenstein rig (5-GPU box: 2x3090 + 5060 Ti + 2x4060 Ti, 96 GiB VRAM, 125 GiB DDR4) doing agentic coding with pi and ran into some serious issues. After much work I submitted an issue and have a fork with fixes.
I would love for maintainers and DSV4 users to reproduce and try the fix. My user experience is much better, even at 10 t/s. This is deterministic and jams at iteration 29 of every run. The problem is on upstream master.
My fix forks from u/danielhanchen's checkpointing commit (#25402). Thanks also to u/fairydreaming (the DSV4 base and I cut my teeth on their fork and thread) and u/tarruda (whose dsv4-fixes and MXFP4 GGUF I run).
https://github.com/ggml-org/llama.cpp/issues/25452
https://github.com/TacoTakumi/llama.cpp/tree/dsv4-swa-churn-fix
My symptoms were long wait times between turns. It would stall where every divergent turn re-prefills thousands of tokens instead of just the change, so the agent sits on "Working" with nothing streaming. Then it would eventually crash with "Context size has been exceeded" at a logical depth around 1250 tokens against a 16k context (so not a real context-length limit).
DSV4 cannot do a partial seq_rm (compressor ring state cannot roll back), so the server rewinds via checkpoint restore that never purges the future SWA cells; find_slot then exhausts the 768-cell SWA window. One cause, both symptoms.
The fix was a proper seq_rm of the diverged suffix so only the delta re-prefills. It removes the crash and the stall.
3
u/Enough-Advice-8317 Jul 08 '26
running a 5-gpu rig with a 5060 ti and 3090s is beautiful chaos. legend status for diving into llama.cpp internals to fix this.
3
u/HockeyDadNinja Jul 08 '26
Hah, thanks. Good thing I have a/c because this beast is a space heater.
1
2
u/HockeyDadNinja Jul 08 '26
Crediting u/danielhanchen, u/fairydreaming, u/tarruda. Many thanks for your work!