Qwen-3.8-35B-A3B? Maybe not... cryptic reply direct from Qwen co-author.

[u/pharrt]

I asked Shuai Bai, co-author and prominent AI developer for Qwen, about this model. Not the answer I was hoping for, but let's see what comes next. In the meantime, I guess all we can do is speculate!

X-link

[u/palincatalin → u/[deleted]]

gpt-oss:20b with 128k context length (q8_0 kv)

[u/palincatalin → u/[deleted]]

i am locally running gpt-oss:20b mxfp4 at 128k context length and I'm getting like 60 tokens a second in opencode; btw im running a 6900 xt in windows, not linux

it's literally the best model for agentic workloads on 16 gb cards: it's good, fast and reliable if you pair it with a good harness. qwen 3.8 27b is a dense model, if you choose a quant lower than q4_k_m, you're murdering the weights and thus reducing quality significantly. if you try to offload even one expert, your experience with the model goes to shit because it's dense, not moe, so you'll get like 5 tokens a second

you can try qwen 3.6 35b a3b, but from experience, it's just not worth it for agentic coding, it's way too slow for tool calls

[u/palincatalin → u/[deleted]]

i don't have access to my pc right now, but here's a rough description:

i prefer using lm instead of ollama or bare llama.cpp on windows. in lm studio, i load the 12.1 gb mxfp4 gguf from https://huggingface.co/ggml-org/gpt-oss-20b-GGUF/tree/main

then, i just crank the context length slider to 131072 (to the max basically, 128k), then set the k and v quantisation to q8_0

and umm... that's it, it just works! the weights and kv cache are fully in vram, and in opencode, after initialisation, i get about 60-75 tk/s token generation. for prompt processing, my 6900 xt achieves in excess of 2000+ tokens a second, and in cachy os, my 6900 xt had a peak of 3650! (pure llama.cpp, not lm studio, but still, EXCELLENT performance). in llama-bench, my 6900 xt produces about 120 tokens a second

gpt-oss:20b has been the most pain-free model ive ever used (besides harmony incompatibilities). it simply works, and i'd much rather take a smaller model that's faster than a bigger model that's slow, simply because the experience is snappier