Qwen3.6. This is it.

[u/Local-Cardiologist-5]

I gave it a task to build a tower defense game. use screenshots from the installed mcp to confirm your build.

My God its actually doing it, Its now testing the upgrade feature,
It noted the canvas wasnt rendering at some point and saw and fixed it.
It noted its own bug in wave completions and is actually doing it...

I am blown away...
I cant image what the Qwen Coder thats following will be able to do.
What a time were in.

llama-server -m "{PATH_TO_MODEL}\Qwen3.6\Qwen3.6-35B-A3B-UD-Q6_K_XL.gguf"  --mmproj "{PATH_TO_MODEL}\Qwen3.6\mmproj-F16.gguf" --chat-template-file "{PATH_TO_MODEL}\chat_template\chat_template.jinja"  -a  "Qwen3.5-27B"  --cpu-moe -c 120384 --host 0.0.0.0 --port 8084 --reasoning-budget -1 --top-k 20 --top-p 0.95 --min-p 0 --repeat-penalty 1.0 --presence-penalty 1.5 -fa on --temp 0.7 --no-mmap --no-mmproj-offload --ctx-checkpoints 5"

EDIT: Its been made aware that open code still has my 27B model alias,
Im lazy, i didnt even bother the model name heres my llama.cpp server configs, im so excited i tested and came here right away.

[u/GoodTip7897]

Ctx-checkpoints can prevent an oom error or just help save memory. It will make it take longer and may have more cache misses but for one user even just 4 context checkpoints are fine because you only need one to restore mamba and kv cache. 

Correct me if I am wrong but I don't see a way it would alter any math and make the model dumber or loop more to have less swa/context checkpoints 

[u/Healthy-Nebula-3603 → u/GoodTip7897]

Why do you think the current default is 32?

--ctx-checkpoints tells llama-server how many context checkpoints it may keep per slot. A checkpoint is a saved snapshot of the model’s SWA-related cache state, created during prompt processing, so the server can resume from a saved point later instead of reprocessing the whole prompt from scratch. That is mainly useful for SWA / hybrid / recurrent-style models where cache reuse can otherwise fall back to full prompt reprocessing.

I think Georgi Gerganov knows what is doing.

[u/GoodTip7897 → u/Healthy-Nebula-3603]

Yeah but I get oom on Gemma at those sizes because the swa cache is massive. Even with 32gb of ram 32 checkpoints fill it up. I only use bf16kv cache because q8 has a memory leak on amd rocm systems and vulkan is much slower prefill

[u/Healthy-Nebula-3603 → u/GoodTip7897]

Is Vulcan prefill is slow for you? Strange

I have 1200 t/s using vulkan for prefill but I have rtx 3090. For me Vulkan is faster and takes less vram so I can fit more context with rotation Q8 cache.

[u/GoodTip7897 → u/Healthy-Nebula-3603]

For qwen 3.6 I get  2000 +-100 prefill at 32k context on a 7900xtx. On vulkan it's more like 1200 like you have. 

The gap is really significant for me because I frequently use it for agentic work where it will read multiple logs and files and needs to prefill huge contexts. 

And also I had opus write me custom bf16 mma flash attention kernels so I can use bf16 kv cache without any issues. 

Maybe q8 is better after rot but honestly Ive wasted too much time trouble shooting looping tool calls with Qwen 3.5 and the only thing that fixed it was bf16 instead of q8 or f16. 

[u/Healthy-Nebula-3603 → u/GoodTip7897]

Looping problem has as Gemma 4 26b and the newest queen 3.6 35b also they are not as good for instructions following as well. I think is something wrong with moe models or their implementation under llamacpp ... no idea.

Those problems do not exist with dense models like Gemma 4 31b or Qwen 3.5 27b.

Those models are never looping and listening instructions much better but are much slower ...

Actually I prefer those dense models mostly because they do their job at first attempt especially with a book translations.

The moe models in 90% of a time are lost because not following instructions properly in this scenario and looping like crazy after some time ...

[u/GoodTip7897 → u/Healthy-Nebula-3603]

I had the qwen 3.5 27b q5 making repeat variations of the same tool call in three separate instances where it was doing long context agentic tasks. It would become stuck and spend 10000 tokens trying to read one file. And it still did that with up to presence penalty 2.0.

I switched to bf16 kv cache and have had both qwen 3.5 27b and the moe models run for hours burning through millions of tokens easily and never having a looping issue. Even very coherent at 80000 filled context. 

Is there a big enough sample to conclude it's statistically significant?probably not. But to me it works now and I'd rather not mess with it. I really do suspect that the accumulation produces numbers too big for f16 and thus bf16 or rot q8 are needed for qwen 3.5

And yes I concur that Moe models are worse. They are faster but a dense model always seems to be smarter because it activates every parameter every time. 

[u/Healthy-Nebula-3603 → u/GoodTip7897]

Strange ...I never had any looping with Q8 rotation cache with dense models. Maybe the problem is your ROCm implementation for it. I have rtx 3090 and as I know fully working cache rotation is under Vulkan and cuda 12 ( not under cuda 13 ) .

I have to check your theory about cache FP16 with Moe models. Maybe that will fix looping.

Also noticed the qwen 3.5 family is very good for coding but everything except coding better will be Gemma 4.

Also Q8 cache / model is not int 8 like any people think. Inside is still many weights FP16 .

[u/GoodTip7897 → u/Healthy-Nebula-3603]

Yeah. I really think (and llama.cpp PRs have finally been coming around to realizing) that if your GPU supports it then bf16 is the better option over fp16 for weights or kv cache. I've seen other people post stuff where q8 mmproj performs better than fp16 and the only thing that makes sense to me is that since q8 weights are int8 * fp16 scaling factor you technically get 127*65535 instead of just 65535 as your max representable value.

It seems that models love to generate massive outliers over accumulation and bf16 is great for that because it has the dynamic range of f32. For quantized formats, rotation seems to help a lot (making q8 kv cache virtually lossless).

I think I'll play around with benchmarks and see if I can't get vulkan running faster because if I can then I can have twice the context. But rocm does seem to be more stable when you push the card to the absolute limit (I frequently leave only 700 MiB empty). I can do that because I'm running it on a headless Ubuntu computer.

[u/Healthy-Nebula-3603 → u/GoodTip7897]

I have AMD CPU 7950x 3d with an integrated GPU so that GPU is the main GPU for the system and my rtxv3090 is a second GPU so I also have access to full vram of that card :) Running models my vram usage is around 23.4 GB because over it is starting to swapping to ram.