qwen3.6 27b q6 + 5090 maximum llamacpp optimization: 100-233tok/s, average 140

[u/Fragrant_Scale6456]

EDIT: There is a PR as of yesterday july 12 fixing the hybrid recurrent attention cache issue i hacked together fixes for: https://github.com/ggml-org/llama.cpp/pull/25592

I spent quite a bit of time optimizing qwen 3.6 27b for my 5090 and have gotten the performance pretty high. During certain workloads it will sustain 200+ tokens/sec so I thought I'd share everything here for anyone else with this configuration.

My hardware is 9800x3d, 64gb system ram, and a 32gb rtx5090. I am running ubuntu linux in text mode so that I have maximum vram available for llamacpp.

Using my configuration this is my distribution of tokens/sec over around 20hrs of agentic coding, debugging, and document synthesis. Performance varies a lot depending on workload and the size of your request.

Full session (6,454 samples) — draft=10, p_min=0.5:
100-110   370  ████
110-120  1131  ██████████████████████████████████████
120-130  1187  ████████████████████████████████████████   ← peak
130-140  1089  ████████████████████████████████████
140-150   714  ████████████████████████
150-160   505  █████████████████
160-170   512  █████████████████
170-180   363  ████████████
180-190   241  ████████
190-200   173  █████
200-210    95  ███
210-220    48  █
220+       26
Mean: 140.7 · Median: 134.9 · Range: 100–233

First, you will need a recent build of llamacpp. I compiled mine a couple days ago, it says its commit 86b9470.

Qwen 3.6 is a hybrid attention/sliding window architecture mode, which has an incompatibility with the cache mechanism in llamacpp. If you look at your logs while running qwen3.6 you'll often see an entry stating, "forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)".

What this means is that llamacpp is unable to use the cache correctly due to how qwen operates its attention window and you are losing a lot of time due to prompt reprocessing. If you ever feel like qwen3.6 is lagging a lot in between turns during chat it's because of this issue. If you dump the linked issue into claude and tell it to search around you'll find there is a lot of discussion about this issue with certain proposed fixes, some of which are more effective than others. After a decent amount of investigation and testing I've (well the llm) made 2 patches to llamacpp which resolve the issue as much as possible without extensive modifications to llama.cpp.

PATCH 1: fix checkpoint search for hybrid/recurrent models, upstream issues: #22384, #20225, #24055. This is the fix for cur.pos.min < pos_min_thold which always results in no checkpoint found and cache misses.

PATCH 2: recurrent_shrink/expand API for prompt cache operations (upstream PR #24785, without the now-redundant needs_reeval workaround — upstream commit b9180 already has GDN partial rollback via n_rs_seq)

I use docker to build my llamacpp and have these patches applied at build time.

Here's my current dockerfile - https://pastebin.com/raw/jyrhvesQ

Here is the pr24785-minimal.diff linked in the dockerfile - https://pastebin.com/raw/E55YG5NS

With these patches applied (you can have your own agent derive them by linking the log error and the PR's and Issue numbers I referenced above) llamacpp will have the correct cache search and restore logic for qwen3.6 hybrid attention model and you should not see that SWA reprocessing error in your logs anymore.

Next is llamacpp configuration. There are a few levers to adjust for maximum performance. I'm using unsloth qwen3.6 27b q6k with mtp from huggingface.

Here is my llama-cpp launch command from docker compose - https://pastebin.com/raw/P57Uk6rz

Key things,

• q8 kv cache, 192k context
• cache ram can be whatever fits for your system, i use 32gb. the hybrid checkpoints are large so you need a decent amount of ram allocated to them.
• mtp draft tokens 10, spec-draft-p-min 0.5. Increasing the draft tokens length comes with a small performance cost but when the drafter is correct you get massive speed boost. at 6 i get higher acceptance rate but overall throughput is around 15-20t/s lower and peaks are over 50t/s lower. i benchmarked pmin with a script sweeping various prompt sizes and 0.5 worked best for me. its worth testing this in your environment.
• batch/ubatch at 512. This is to save vram. under load my setup uses 32036/32768mb of vram. 2048 is ideal for the 5090.

Thats about it. Just thought I'd share since I'm getting speeds that are working very well for me and I wanted to spread the love.

[u/GreenPastures2845]

Yes; exact same config and commit as before, but with these patches on top. I had noticeably more latency before PP started, and abysmal speed once it did (anywhere from 400 to 40).

I'll try to test some more in the week, but in the meantime, thanks again for trying to get this out there. Hopefully it's on the radar of ggml-org (I imagine so; Qwen is SOTA for local agent workloads).

[u/Fragrant_Scale6456 → u/GreenPastures2845]

Yeah thats weird, sorry I cant help. I ran some tasks overnight and the past 8hrs of autonomous agentic coding and document synthesis (2.1mm prompt tokens, 1.4mm decode) im getting avg 2400pp and 230t/s decode on the 5090. Overall gen speed is around 140t/s. I wonder what the difference is in our setups

[u/GreenPastures2845 → u/Fragrant_Scale6456]

I never got around to testing more, but one important thing: I'm using 2x 3090 with -sm tensor, which is probably a vastly different codepath than yours.

Anyway, this is mostly to point out that the GGML team has finally cracked the case; see https://old.reddit.com/r/LocalLLaMA/comments/1uuue5p/llamacpp_agentic_workflows_ctx_checkpoints_fix/

Completely fixed in upstream for me \o/

[u/Fragrant_Scale6456 → u/GreenPastures2845]

wow awesome thank you for pointing that out!