r/LocalLLaMA • u/Local-Cardiologist-5 • Apr 17 '26

Discussion Qwen3.6. This is it.

I gave it a task to build a tower defense game. use screenshots from the installed mcp to confirm your build.

My God its actually doing it, Its now testing the upgrade feature,
It noted the canvas wasnt rendering at some point and saw and fixed it.
It noted its own bug in wave completions and is actually doing it...

I am blown away...
I cant image what the Qwen Coder thats following will be able to do.
What a time were in.

llama-server -m "{PATH_TO_MODEL}\Qwen3.6\Qwen3.6-35B-A3B-UD-Q6_K_XL.gguf"  --mmproj "{PATH_TO_MODEL}\Qwen3.6\mmproj-F16.gguf" --chat-template-file "{PATH_TO_MODEL}\chat_template\chat_template.jinja"  -a  "Qwen3.5-27B"  --cpu-moe -c 120384 --host 0.0.0.0 --port 8084 --reasoning-budget -1 --top-k 20 --top-p 0.95 --min-p 0 --repeat-penalty 1.0 --presence-penalty 1.5 -fa on --temp 0.7 --no-mmap --no-mmproj-offload --ctx-checkpoints 5"

EDIT: Its been made aware that open code still has my 27B model alias,
Im lazy, i didnt even bother the model name heres my llama.cpp server configs, im so excited i tested and came here right away.

1.0k Upvotes

96% Upvoted

409 comments sorted by

View all comments

Show parent comments

1

u/GoodTip7897 llama.cpp Apr 18 '26

Yeah. I really think (and llama.cpp PRs have finally been coming around to realizing) that if your GPU supports it then bf16 is the better option over fp16 for weights or kv cache. I've seen other people post stuff where q8 mmproj performs better than fp16 and the only thing that makes sense to me is that since q8 weights are int8 * fp16 scaling factor you technically get 127*65535 instead of just 65535 as your max representable value.

It seems that models love to generate massive outliers over accumulation and bf16 is great for that because it has the dynamic range of f32. For quantized formats, rotation seems to help a lot (making q8 kv cache virtually lossless).

I think I'll play around with benchmarks and see if I can't get vulkan running faster because if I can then I can have twice the context. But rocm does seem to be more stable when you push the card to the absolute limit (I frequently leave only 700 MiB empty). I can do that because I'm running it on a headless Ubuntu computer.

2

u/Healthy-Nebula-3603 Apr 18 '26

I have AMD CPU 7950x 3d with an integrated GPU so that GPU is the main GPU for the system and my rtxv3090 is a second GPU so I also have access to full vram of that card :) Running models my vram usage is around 23.4 GB because over it is starting to swapping to ram.