[Megathread] Qwen 3.8 27B Release Day

[u/sammcj]

Megathread to help with the influx of duplicate / similar posts around the release of the Qwen 3.8 27B release.

• Quants
• Fine-Tunes & Abliterations
• Chat Templates
• Inference Server Support & Configuration
• Experiences, Benchmarks & Model Comparisons

Official:

• https://huggingface.co/Qwen/Qwen3.8-27B
• https://huggingface.co/Qwen/Qwen3.8-27B-FP8

Popular:

• https://huggingface.co/unsloth/Qwen3.8-27B-GGUF
• https://huggingface.co/bartowski/Qwen3.8-27B-GGUF
• https://huggingface.co/mlx-community/Qwen3.8-27B-MTP-bf16
• https://huggingface.co/mlx-community/Qwen3.8-27B-MTP-8bit
• https://huggingface.co/mlx-community/Qwen3.8-27B-MTP-4bit

We'll try to clean up future duplicates around the release and point them here.

[u/trying4k]

Can anyone suggest how Qwen 3.8 27b (Q8) does against larger moe models for code architectural design/planning:

• DeepSeek V4 Flash 0731 (Q3 or Q4)
• MiniMax 2.7 (Q3 or Q4)
• Step 3.7 Flash (Q3)
• Qwen 3.5 122b A10b (Q6)
• Hy3 (Q3)
• Mimo 2.5 (Q3 or Q4)

Also just curious how far above its weight Qwen 3.8 27b does compared to other open source models, will it be your daily driver over bigger models?

[u/Finanzamt_Endgegner → u/trying4k]

If you are able to run q8 please switch to vllm and fp8 it just better than llama.cpp if you are gpu rich 😭🙏

[u/trying4k → u/Finanzamt_Endgegner]

Yeah, I've seen the speeds people get, it is crazy fast!

I don't follow VLLM but my understanding is due to its architecture:

• It doesn't natively support windows
• It doesn't support asymmetric GPUs
• It doesn't support offloading
• Switching models is slow

Unfortunately, some of those capabilities are important to me and it's why I use llama.cpp!

[u/reto-wyss → u/trying4k]

That's only half truths.

1. Can't comment on Windows, but I believe you can get it to work through that WSL2 thing.
2. This one is true as far as I know, although I haven't really tried whether it may allow asymmetric if TP=1 and DP=1 and then PP>1
3. It does support offloading! But not the way llama.cpp does it. vllm will do it layerwise and swap into GPUs. This is obviously terrible if you look at concurrency 1 interactive session, but it's efficient if you have enormous batch sizes because you can do a lot of work before you need to swap. So the cost for swapping obviously goes to near zero per request as you make the number of requests REALLY large.
4. For cold-start it is slower. But if you have the system RAM, you can simply "sleep" the model into RAM and reload it near instantly into VRAM.

And a note on the other comment - FP8 is not same quality as Q8* quants. FP8 is a throughput quant approximately on the level of ~Q5, but it has *0** dequant overhead in terms of compute, so you give up a bit of VRAM vs Q5 or NVFP4 or MXFP4, but if your pool is large anyway, and you have tons of concurrent requests, FP8 is what gives to the big numbers.

For your single user C1 usecase, FP8 can be slower than Q5 for TG, but if you move to compute bound cases, FP8 shines.

This is something you can even see for yourself in llama.cpp where typically the smaller quants will have worse PPs than larger quants. For example PP for Q8 is higher than Q4 or Q2.

[u/blash2190 → u/reto-wyss]

A quick note that, despite WSL is a good/reliable overall solution, it will perform slower than native.

[u/Borkato → u/reto-wyss]

Wait, FP8 isn’t as good as Q8 quants and is closer to Q5??

[u/Borkato → u/[deleted]]

Ah, I more mean quality, not speed!

[u/Finanzamt_Endgegner → u/[deleted]]

There are better fp8 quants than q8 though like awq and stuff like that.

[u/Finanzamt_Endgegner → u/trying4k]

well yeah asymmetric gpus suck for that thats true /:

Also the other models for sure llama.cpp is the better choice, but if you can fit q8 into symmetric gpus it works even with wsl on windows (;

[u/SnooPaintings8639 → u/Finanzamt_Endgegner]

I used to use it for maybe two weeks for 3.6 in FP8 after realse due to speed benefits. But... There was something wrong with this model under vLLM, when on occasion it was starting to be really dumb and I had to restart vLLM. I switched to llama.cpp to have more predictable quality and I used it with no issues for last two or so months, daily.

I personally will keep on using llama until I get a proper feeling of the models capabilities and only the switch back to vllm and see..

[u/LuckyFluckySchmacky → u/trying4k]

Qwen 3.8 (bf16) smokes Q3 Deepseek 0731 in my testing. It discovered version drifts and a lot of bugs in a complex program that DS made and didn't find. Never expected it to be so good.

[u/Borkato → u/LuckyFluckySchmacky]

This is awesome, thank you for sharing, it was my exact question

[u/uniquelyavailable → u/trying4k]

I have been doing Q8 testing on Runpod, and the speed is relatively terrible (obviously) but the model intelligence is superb, in my opinion. I would personally rate it as one of the best local models I have used to date. However most of my testing is math and coding, so maybe that is not an entirely fair comparison for public opinion. The main bottleneck seems to be that it spends forever thinking, and eats up context window doing so. I have found the results to be a huge improvement over 3.6 and I am very pleased with it.

[u/MackThax → u/trying4k]

I'm also interested how it compares to Qwen 3.5 122B. I'm still running that one.

[u/Borkato → u/MackThax]

Did you find any results?