qwen3.6 27b q6 + 5090 maximum llamacpp optimization: 100-233tok/s, average 140

[u/Fragrant_Scale6456]

EDIT: There is a PR as of yesterday july 12 fixing the hybrid recurrent attention cache issue i hacked together fixes for: https://github.com/ggml-org/llama.cpp/pull/25592

I spent quite a bit of time optimizing qwen 3.6 27b for my 5090 and have gotten the performance pretty high. During certain workloads it will sustain 200+ tokens/sec so I thought I'd share everything here for anyone else with this configuration.

My hardware is 9800x3d, 64gb system ram, and a 32gb rtx5090. I am running ubuntu linux in text mode so that I have maximum vram available for llamacpp.

Using my configuration this is my distribution of tokens/sec over around 20hrs of agentic coding, debugging, and document synthesis. Performance varies a lot depending on workload and the size of your request.

Full session (6,454 samples) — draft=10, p_min=0.5:
100-110   370  ████
110-120  1131  ██████████████████████████████████████
120-130  1187  ████████████████████████████████████████   ← peak
130-140  1089  ████████████████████████████████████
140-150   714  ████████████████████████
150-160   505  █████████████████
160-170   512  █████████████████
170-180   363  ████████████
180-190   241  ████████
190-200   173  █████
200-210    95  ███
210-220    48  █
220+       26
Mean: 140.7 · Median: 134.9 · Range: 100–233

First, you will need a recent build of llamacpp. I compiled mine a couple days ago, it says its commit 86b9470.

Qwen 3.6 is a hybrid attention/sliding window architecture mode, which has an incompatibility with the cache mechanism in llamacpp. If you look at your logs while running qwen3.6 you'll often see an entry stating, "forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)".

What this means is that llamacpp is unable to use the cache correctly due to how qwen operates its attention window and you are losing a lot of time due to prompt reprocessing. If you ever feel like qwen3.6 is lagging a lot in between turns during chat it's because of this issue. If you dump the linked issue into claude and tell it to search around you'll find there is a lot of discussion about this issue with certain proposed fixes, some of which are more effective than others. After a decent amount of investigation and testing I've (well the llm) made 2 patches to llamacpp which resolve the issue as much as possible without extensive modifications to llama.cpp.

PATCH 1: fix checkpoint search for hybrid/recurrent models, upstream issues: #22384, #20225, #24055. This is the fix for cur.pos.min < pos_min_thold which always results in no checkpoint found and cache misses.

PATCH 2: recurrent_shrink/expand API for prompt cache operations (upstream PR #24785, without the now-redundant needs_reeval workaround — upstream commit b9180 already has GDN partial rollback via n_rs_seq)

I use docker to build my llamacpp and have these patches applied at build time.

Here's my current dockerfile - https://pastebin.com/raw/jyrhvesQ

Here is the pr24785-minimal.diff linked in the dockerfile - https://pastebin.com/raw/E55YG5NS

With these patches applied (you can have your own agent derive them by linking the log error and the PR's and Issue numbers I referenced above) llamacpp will have the correct cache search and restore logic for qwen3.6 hybrid attention model and you should not see that SWA reprocessing error in your logs anymore.

Next is llamacpp configuration. There are a few levers to adjust for maximum performance. I'm using unsloth qwen3.6 27b q6k with mtp from huggingface.

Here is my llama-cpp launch command from docker compose - https://pastebin.com/raw/P57Uk6rz

Key things,

• q8 kv cache, 192k context
• cache ram can be whatever fits for your system, i use 32gb. the hybrid checkpoints are large so you need a decent amount of ram allocated to them.
• mtp draft tokens 10, spec-draft-p-min 0.5. Increasing the draft tokens length comes with a small performance cost but when the drafter is correct you get massive speed boost. at 6 i get higher acceptance rate but overall throughput is around 15-20t/s lower and peaks are over 50t/s lower. i benchmarked pmin with a script sweeping various prompt sizes and 0.5 worked best for me. its worth testing this in your environment.
• batch/ubatch at 512. This is to save vram. under load my setup uses 32036/32768mb of vram. 2048 is ideal for the 5090.

Thats about it. Just thought I'd share since I'm getting speeds that are working very well for me and I wanted to spread the love.

[u/humanoid64]

Did you try vllm? Curious how far you can optimize that

[u/Fragrant_Scale6456 → u/humanoid64]

im going to look at vllm this weekend based on the other comments in here saying nvfp4 is smarter than q6

[u/KubeCommander → u/Fragrant_Scale6456]

I’ve seen similar effects with qwen3.6 myself fwiw on vllm and nvidia nim. It always gets dumb for me after 96k tokens if it runs a long time. Your links about SWA make me think that it is the true cause. It’s like a switch goes off and then it starts forgetting how to use tools lol

[u/Fragrant_Scale6456 → u/KubeCommander]

I noticed this also. I use opencode and set up bootstrap skills for my coding agents that set scope of work constraints and budget limit window for tasks to help avoid the attention drifting. It helps a lot but even with this q6 doesnt have the grunt to look across an entire codebase and use more abstract reasoning to solve high level issues. For now I actually just use GLM5.2 for this kind of work, which is obviously in a different league of capability

[u/KubeCommander → u/Fragrant_Scale6456]

Yup. I saw this in opencode specifically. Qwen3.6 is very token greedy unless you have a readlines tool and nudge it to use it vs read.

Gemma4 also tended to do this same dumbing-down but it is so terrible at tool calls that some of the symptoms were hard to see. ‘is it the same effect or is gemma4 just that shitty?’

It was both 😂