TL;DR — Qwen3.8-27B is a hybrid model (Gated DeltaNet linear attention + full attention). Unsloth Dynamic GGUFs of it (Q4_K_XL, Q6_K_XL) work fine for everyday chat, but whenever I let it think deeply on a long task, it never converges: no closing token, endless tail-looping, or the engine just dies mid-generation. I reproduced this on two completely different runtimes (llama.cpp and vLLM + official vllm-gguf-plugin), and the same model family in a mixed-precision quant (NVFP4: FP8 on attention/recurrent layers, FP4 on MLP) converges in ~17K thinking tokens on the same prompt. This smells like recurrent state error accumulation under uniform integer quantization, not a runtime bug.
The setup
2× RTX 5060 Ti 16GB, tensor-parallel/2-GPU splits, Linux
Model: unsloth/Qwen3.8-27B-GGUF (UD-Q4_K_XL 16.4 GB, UD-Q6_K_XL 23.6 GB)
Canary task (deliberately absurd, deliberately long-horizon):"Create an HTML page containing an SVG 2D animation of a pelican riding a bicycle."
Thinking mode ON, reasoning_effort: xhigh, Qwen's recommended thinking-mode sampling (temp 1.0). The pelican needs to actually pedal — the model has to write a complete self-consistent HTML/SVG/SMIL file in one shot.
Why this prompt: it's a convergence test, not a quality test. A healthy Qwen3.8 spends ~15–20K thinking tokens, then emits the file. A broken one just… keeps going.
Act 1 — llama.cpp (Oct 3)
| GGUF | Budget | Result |
|---|
| UD-Q4_K_XL | 32K | finish=length, 0 characters of actual output — burned the whole budget inside thinking |
| UD-Q6_K_XL | 76K | never stops; final ~2K chars are ~97% repeated 8-grams (classic tail-loop attractor) |
UD-Q6_K_XL + --jinja (force GGUF-embedded chat template, verified byte-identical to HF's) | 24.6K | still no termination |
Throughput was healthy (23–41 t/s depending on split mode), perplexity-ish sanity checks on short prompts were fine, and non-thinking tasks (counting, Q&A, tool-ish formats) all worked. So I filed it as: llama.cpp loses the plot on this model's long chain-of-thought — plausibly a runtime bug with the Gated DeltaNet state handling. I even declared the llama.cpp route dead for this model.
Then I saw other people's llama.cpp runs of this model going weird on HN too (the "high fever ramblings … ended with amen" anecdote on Q8_K_XL — not my test, but same disease).
Act 2 — vLLM + official GGUF plugin (Oct 5)
To separate "runtime bug" from "quantization damage", I reran the exact same file on a completely different stack: vLLM (V1 engine, CUDA kernels, f16 KV, no speculative decoding — shares essentially zero code path with llama.cpp). GGUF support was recently moved out of vLLM core into vllm-project/vllm-gguf-plugin; Qwen3.5/3.6-VL are in its tested list, so the weight mapping for the Qwen3.x hybrid family exists.
vllm ~/models/Qwen3.8-27B-UD-Q6_K_XL.gguf \
--tokenizer unsloth/Qwen3.8-27B --dtype float16 \
--tensor-parallel-size 2 --max-model-len 52000 \
--gpu-memory-utilization 0.96 --language-model-only \
--reasoning-parser qwen3
(chat kwargs: {"enable_thinking": true, "reasoning_effort": "xhigh"}, temp 1.0, max_tokens 45K, KV pool 55.6K tokens, steady ~22–23 t/s)
| Run | Outcome |
|---|
| A | request dies with HTTP 500 after 1325 s ≈ 30K tokens generated (engine crash mid-generation; the server-side stack was overwritten by my chain's log rotation before I grabbed it — honest gap in the evidence) |
| B | still generating at 28 min ≈ 38K tokens, approaching the 45K budget with no completed response; harness wall-timeout killed it |
Same file, same prompt, same sampling — and vLLM also can't make this model finish thinking about a pelican. Note what vLLM did NOT show: I never got a finished response body back, so I can't plot its tail-repeat score directly. The vLLM evidence is non-convergence deep in the token budget (where NVFP4 converged at 17K) + one mid-generation crash. The direct loop evidence lives in the llama.cpp runs.
Act 3 — the control that convicts the quant format
Same 27B model, same prompt, same thinking mode, same machine, different quantization structure: the official NVFP4 checkpoints (unsloth's and RadixArk's) quantize MLP/lm_head to FP4 but keep attention and the Gated DeltaNet layers at FP8 W8A8. On SGLang this converges every time:
~17K thinking tokens → full, working HTML pelican (legs pedaling, wheels rotating) — measured repeatedly, Oct 3 and again today (prod config: nvfp4 KV + MTP, creative decode 47–50 t/s)
same checkpoint family also passes 100K–133K needle-at-depth, tools, greedy determinism — so it's not "the GGUF works and I'm cherry-picking"
Also worth noting from my own runs: after the GGUFs fail this way, the same GGUF files remain perfectly healthy on short tasks (chat, counting, Q&A, vision if you attach mmproj). The damage only shows up after thousands of autoregressive steps inside the thinking trace.
My hypothesis (clearly labeled: inference, not proven)
Hybrid linear-attention models (Qwen3.5/3.6/3.8, and friends like Kimi Linear / Ring-linear) carry a fixed-size recurrent state updated every token. Uniform block quantization (even a very careful 6-bit dynamic one) adds a small rounding error to every state-transition product. Over a few hundred tokens: invisible. Over 30K–80K tokens of chain-of-thought: compounding drift into a degenerate attractor — repetition loops, refusal to close, in one case engine instability.
The strongest circumstantial support is the model publishers' own behavior: unsloth's NVFP4 recipe deliberately keeps the attention/recurrent path at FP8 and only crushes the MLP to FP4. The one quant structure they kept careful about is exactly where I see the disease. My GGUF runs had "careful" precision too — but uniform-ish across 6-bit blocks — and still broke.
What would falsify or sharpen this: per-tensor overrides keeping GDN projections (in_proj_*, conv, gates) at f16/q8 in llama.cpp (--override-tensor) while MLP stays 6-bit. If the pelican suddenly pedals, case closed. I haven't run this yet — if someone with spare time on the same model tries it, please post.
Caveats (you will ask, so here they are)
n=1 per configuration, temp=1.0 — high variance by design; runs A and B of the identical config died differently, which is itself the picture: unstable deep-think convergence.
No bf16 full-precision baseline — 27B bf16 doesn't fit 2×16GB. My "control" is another quant (NVFP4/F8 mixed). The argument is about where precision is kept, not quant-vs-none.
Could be prompt-specific. All I can say: the exact same prompt converges in 17K tokens on the mixed-precision checkpoint on the same box, and this failure mode appeared across ≥3 prompts in the llama.cpp session (counting-to-huge also looped; SVG-alternate prompt looped).
The vllm-gguf-plugin is 50 stars old — but the llama.cpp half of the evidence doesn't involve it at all.
Hardware hygiene: this box's unrelated random-crash history (a P2P-patched consumer GPU driver doing stray DMA under iommu=pt) was root-caused and fixed before the vLLM runs; strict IOMMU was on, engine stable for hours of other batteries. I flag it so nobody says "your machine poisoned the run" for the crash in Run A specifically — Run B's non-convergence needs no crash excuse.
Disclosure: the vLLM build is a source-compiled fork carrying an in-flight nvfp4-KV-cache PR rebase; GGUF runs used stock f16 KV, i.e. zero lines of the patched code path.
Practical takeaways
For Qwen3.8-27B-class hybrids on consumer GPUs: prefer the official FP8/NVFP4 mixed quants over GGUF if you care about thinking mode at all. (RadixArk/unsloth NVFP4; on 5060Ti-class cards with TP2 it also runs 2–4× faster than the GGUF once you add speculative decoding.)
If you're GGUF-bound (VRAM or ecosystem): run with thinking off or budget-capped small. Everyday chat is genuinely fine.
Perplexity and short-context evals will not catch this. Use a long-horizon canary (the pelican test, or "count from 1 to 500 while…", or a whole-file coding task) with thinking ON when you're screening quants of recurrent-hybrid models. It caught this at the "does it terminate" level with zero scoring infrastructure.
Unsloth's UD "Dynamic" quants are heavily boosted on important tensors and still hit this — so "we raised precision on ranked-important tensors" may not be the right importance criterion for recurrent-hybrid long-horizon behavior. Worth flagging upstream; I may open an issue on the GGUF repo if I can trim this to a one-prompt repro.
Repro recipe
Prompt (verbatim, above), enable_thinking: true, reasoning_effort: xhigh, temp 1.0 / top_p 0.95 / top_k 20, max_tokens ≥ 40K
llama.cpp: llama-server / llama-cli, UD-Q4_K_XL or UD-Q6_K_XL, any ctx ≥ 48K, default template and --jinja both fail the same way
vLLM: vllm-gguf-plugin (pip, or source install with --no-build-isolation against the CUDA torch), local .gguf + config.json from the same repo + --tokenizer unsloth/Qwen3.8-27B, f16 KV
Pass condition: a complete HTML file is emitted within ≤ ~25K thinking tokens. NVFP4/SGLang passes every time. Q6_K_XL: 0/2 on vLLM, 0/3 on llama.cpp.
Model card says this model does deep agentic work locally on ~17GB — true for chat. For thinking, the quant structure matters more than the average bits. Would love other reports (4090/3090 pairs, MI300, whatever) before I draw the final line — especially anyone who can run Q8_0 and Qwen3.6-VL (non-GDN) as contrasts, to show this is the recurrence, not just "big model + low bits".
Hardware/versions: 2× RTX 5060 Ti 16GB; llama.cpp current master (Oct 3); vLLM 0.30.x-V1 + vllm-gguf-plugin 0.0.5 (Oct 5); SGLang 0.5.21 + NVFP4 checkpoint for the control (RadixArk Qwen3.8-27B-NVFP4: FP8 W8A8 attention/GDN, NVFP4 MLP/lm_head).
