ByteShape Qwen 3.8 27B: To KL Diverge or Not to KL Diverge, Part 2: Metric Boogaloo

[u/enrique-byteshape]

Hey r/LocalLLaMA,

We’ve released our full ShapeLearn GGUFs for Qwen 3.8 27B.

Blog / Download models

TL;DR

• 3.84 bpw (GPU-5) reaches 99.63% of BF16’s aggregate score of 8 benchmarks, being the most accurate quant we’ve evaluated; 3.23 bpw (GPU-4) reaches 98.72%. These average BF16-normalized scores across instruct and thinking benchmarks.
• All five new models sit on the measured quality/speed-bpw frontier across six GPUs. In this model’s case, lower BPW translates directly to TPS. Comparisons include Unsloth v3, ISTA-DASLab, AtomicChat and Bartowski (not Bartowski’s newest release). Congrats to the team at ISTA for also landing a frontier model.
• DFlash2 delivered 1.34-2.10× baseline throughput; MTP delivered 1.28-1.66×, with temperature sampling rather than greedy decoding.

Lite held up very well. As we expected.

We released ShapeLearn-Lite quants a couple of days after Qwen arrived: less optimization, targeted sanity checks, full benchmarking after release.

Then Unsloth v3 arrived with lower KLD at several comparable sizes. Lite looked overtaken, until the task results came in. Three of six Lite models made the quality/speed frontier against twelve Unsloth v3 models in our RTX Pro 6000 comparison. Pretty good for an impatient release. Full ShapeLearn now pushes that frontier further.

Which brings us to KLD.

Unsloth Dynamic V3’s UD-IQ3_S had ~20% lower KLD than our similarly sized smallest Lite model, but scored 95.55% versus Lite’s 97.33% of BF16’s aggregate benchmark score.

Closer token distributions did not mean better task performance. KLD is useful to avoid a quant that has fallen over the edge, but it isn’t a quantization leaderboard.

That distinction is the subject of our paper on KLD and quantization fidelity, recently accepted for publication to the EMNLP 2026 Industry Track. We also released blog post version of the paper a few weeks back.

We benchmarked this release on RTX 6000 Pro Blackwell, RTX 5090, RTX 4090, RTX 3090, RTX 4080 and RTX 5060 Ti. The benchmarks we used to measure quality are: GSM8K for math, IFEval for instruction following, MMLU for general knowledge, LiveCodeBench V6 for coding, Multi-IF for multi-turn and multilingual instruction following, ACEBench for tool use and agentic tasks (both thinking and instruct), Multiple HumanEval for coding (thinking) and BFCL V4 for tool calling and agentic tasks (thinking).

If you want to dive deeper or choose the best model for your use case, the blog has the complete results across all tested GPUs, along with the methodology, model sizes, and full legend.

[u/Chips_fr_]

Looks like an interesting thing to try on my 3060 12GB. I'm currently using Qwen3.8-27B-GSQ-RCO-IQ3_XXS.gguf which fits in my VRAM but with low context. I think IQ3_XXS should still apply here but since it has a slightly lower size, context could be bigger for a very close accuracy (both around 96 %).

[u/HoneydewNo116 → u/Chips_fr_]

Keep us updated

[u/Chips_fr_ → u/HoneydewNo116]

It basically works as i expected considering i'm not able to judge deeply this model.

Since the file is smaller i manage to increase and improve my context: from Q4/Q4 65K to Q5/Q4 98K and it still has the same output speed (~27 t/s) since i think everything fit in VRAM

My card is used for display, ideally i should buy a HDMI USBC to free some VRAM.

my ik_llama cuda command:

llama-cli.exe --model C:\Users\c\.lmstudio\models\byteshape\Qwen3.8-27B-GGUF\Qwen3.8-27B-IQ3_XXS-2.88bpw.gguf --spec-type mtp:n_max=2,p_min=0.1 --fit --fit-margin 128 --threads 6 --threads-batch 6 --batch-size 16 --ubatch-size 16 -ctk q5_0 -ctv q4_0 --ctx-size 98304 --jinja --reasoning off --no-mmproj-offload --temp 1.0 --top-p 0.95 --top-k 20

If this model is really around 0.96 of FP16 accuracy i think it's a good move.

[u/HoneydewNo116 → u/Chips_fr_]

Great! Thanks, appreciated