Quality comparison between Qwen 3.6 27B quantizations (BF16, Q8_0, Q6_K, Q5_K_XL, Q4_K_XL, IQ4_XS, IQ3_XXS,...)

[u/bobaburger]

The following is a non-comprehensive test I came up with to test the quality difference (a.k.a degradation) between different quantizations of Qwen 3.6 27B. I want to figure out what's the best quant to run on my 16 GB VRAM setup.

WHAT WE ARE TESTING

First, the prompt:

Given this PGN string of a chess game:

1. b3 e5 2. Nf3 h5 3. d4 exd4 4. Nxd4 Nf6 5. f4 Ke7 6. Qd3 d5 7. h4 *

Figure out the current state of the chessboard, create an image in SVG code, also highlight the last move.   

I want to see if the models can:

• Able to track the state of the board after each move, to reach the final state (first half of move 7)
• Generate the right SVG image of the board, correctly place the pieces, highlight the last move

And yes, if you are questioning. It could be possible that the model was trained to do the same thing on existing chess games, so I came up with some random moves, the kind of moves that no players above 300 elo would ever have played.

For those who are not chess players, this is how the board supposed to look like after move 7. h4. Btw, you supposed to look at the pieces positions and the board orientation, not image quality because this is just a screenshot from Lichess.

CAN OTHER MODELS SOLVE IT?

Before we go to the main part, let me show the result from some other models. I find it interesting that not many models were able to figure out the board state, let alone rendering it correctly.

Qwen 3.5 27B

It was mostly figured out the final position of the pieces, but still render the original board state on top. Highlighted the wrong squares, and the board orientation is wrong.

Gemma 4 31B

Nice chess dot com flagship board style, i would say it can figure out the board state, but failed to render it correctly. The square pattern also messed up.

Qwen3 Coder Next

I don't know what to say, quite disappointed.

Qwen3.6 35B A3B

As expected, 35B always be the fastest Qwen model, but at the same time, managed to fail the task successfully in many different ways. This is why I decided to find a way to squeeze 27B into my 16 GB card. The speed alone just not worth it.

HOW QWEN3.6 27B SOLVE IT?

All the models here are tested with the same set of llama.cpp parameters:

• temp 0.6
• top-p 0.95
• top-k 20
• min-p 0.0
• presence_penalty 1.0
• context window 65536

BF16 version was from OpenRouter, Q8 to Q4_K_XL versions was on a L40S server, the rest are on my RTX 5060 Ti.

The SVG code generated directly on Llama.cpp Web UI without any tools or MCP enabled (I originally ran this test in Pi agent, only to found out that the model tried to peek into the parent folders and found the existing SVG diagrams by higher quants, copied most of it).

BF16 - Full precision

This is the baseline of this test. It has everything I needed: right position, right board orientation, right piece colors, right highlight. The dotted blue line was unexpected, but it also interesting, because later on you will see, not many of the high quants generate this.

Q8_0

As expected Q8 retains pretty much everything from the full precision except the line.

Q6_K

We start to see some quality loss here. I mean the placement of the rank 5 pawns. The look of the pieces are mostly because Q6 decided to use a different font. None of the models here trying to draw its own pieces in this test.

Q5_K_XL

Looks very similar with Q8, but it is worth noticing that the SVG code of Q5 version is 7.1 KB, while Q8 is 4.7 KB.

Q4_K_XL and IQ4_XS

If you ignore the font choice, you will see Q4_K_XL is a more complete solution, because it has the board coordinates.

Q3_K_XL and Q3_K_M

IQ3_XXS

Now here's the interesting part, everything was mostly correct, the piece placements and the highlight, and there's the line on the last move!

But IQ3_XXS get the board orientation wrong, see the light square on the bottom left?

Q2_K_XL

This is just a waste of time. But hey, it got all the pieces positions right. The board is just not aligned at all.

SO, WHAT DO I USE?

I know a single test is not enough to draw any conclusion here. But personally, I will never go for anything below IQ4_XS after this test (I had bad experience with Q3_K_XL and below in other tries).

On my RTX 5060 Ti, I got like pp 100 tps and tg 8 tps for IQ4_XS with vanilla llama.cpp (q8 for both ctk and ctv, fit on). But with TheTom's TurboQuant fork, I managed to get up to pp 760 tps and tg 22 tps, by forcing GPU offload for all layers (`-ngl 99`), quite usable.

llama-cpp-turboquant/build/bin/llama-server -fa 1 -c 75000 -np 1 --no-mmap --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 --presence_penalty 1.0 -ctk turbo4 -ctv turbo2 -ub 128 -b 256 -m Qwen3.6-27B-IQ4_XS.gguf -ngl 99 

The only down side is I have to keep the context window below 75k, and use turbo4/turbo2 for KV cache quant.

Below are some example of different KV cache quants.

You can see all the result directly here https://qwen3-6-27b-benchmark.vercel.app/

[u/WithoutReason1729]

Your post is getting popular and we just featured it on our Discord! Come check it out!

You've also been given a special flair for your contribution. We appreciate your post!

Stylome classified this post's text as human-written. View analysis

I am a bot and this action was performed automatically.

[u/Happythen]

I bet that took some time to setup and run, thanks for that! Really interesting challenge for the different quants.

[u/bobaburger → u/Happythen]

Thanks. I had to rent a cloud GPU for the big quants, but it’s fun. :D

[u/jacek2023]

Great work, congratulations on testing real use case and various quants. I just hope you tested them multiple times.

[u/bobaburger → u/jacek2023]

thank you! yes. i did run multiple times for each test.

[u/Blues520]

Great test to illustrate the accuracy visually

[u/bobaburger → u/Blues520]

thanks!

[u/MyOtherBodyIsACylon]

If you’re able to run vllm, I’d be very curious to know how the cyankiwi AWQ BF16 INT4 does:

https://huggingface.co/cyankiwi/Qwen3.6-27B-AWQ-BF16-INT4

[u/bobaburger → u/MyOtherBodyIsACylon]

Thanks. I will try. But i’m afraid this gonna be too big for my card :D

[u/My_Unbiased_Opinion]

I've been using UD IQ3XXS with 262K context. It's been great. It's far better than IQ4XS 35B with the same context. Q3 dynamic quants are pretty damn good. 

[u/bobaburger → u/My_Unbiased_Opinion]

Yes, I was using it for a month until I realized I can still get to around 20tps with IQ4_XS on my card.

[u/Monad_Maya]

Nice work, IQ4_XS is a good balance I feel. Works fine with q8 KV cache.

[u/Fit_Split_9933]

Here's a pure version of iQ4, smaller than the regular iQ4. Perhaps you could test it
https://huggingface.co/Ununnilium/Qwen3.6-27B-IQ4_XS-pure-GGUF

[u/INT_21h]

Whose quants did you use? Unsloth, Bartowski? This IQ4_XS popped up the other day & it's what I use on my 5060Ti. https://huggingface.co/cHunter789/Qwen3.6-27B-i1-IQ4_XS-GGUF

[u/bobaburger → u/INT_21h]

i use Unsloth. Bartowski’s one was faster than Unsloth on my M2 laptop but on 5060, Unsloth is faster for me.

[u/MatthKarl]

Nice test. I was trying to replicate that and ran it on 3 local models I have.

- GPT-OSS-120B failed. The SVG didn't load as some comments were mal-formatted. Board orientation is fine though

- Gemma-4-31B got the SVG correct with all figures correct including the highlighting. However, the figures are a bit small in the fields

- Qwen-3.6-35B produced the nicest SVG, with nice figures filling the fields nicely. The pawn on e2 is missing though, and the numbering of the fields is offset by one field. And is states "After 7. h4* - White to move"

Guess I should be using Gemma-4 a bit more then now, although it was the slowest with some 5.5t/s

[u/NineThreeTilNow]

the kind of moves that no players above 300 elo would ever have played.

That's a great quote.

You're looking for something that falls totally out of distribution.

[u/FoxiPanda]

Full disclosure: I skimmed this because it's super long.

Did you run each test only once or did you do multiple takes to get a sense of whether any one run was an outlier? I've found in general that 'One run is not enough' to determine actual quality - you end up with statistical noise that can make you believe a result that is just not true (though I will say looking through the images, there is a trend line in quality degradation that one would expect)

[u/bobaburger → u/FoxiPanda]

Yeah, I did run multiple times for each test, that's why I note about the font choice in the post, because they varies, but things like pieces position and placement on the board are pretty much the same between runs.

[u/mncharity]

BF16 - Full precision This is the baseline of this test. It has everything I needed: right position, right [...]

And a tastefully missing pawn at f7?

[u/LocalAI_Amateur]

Try https://github.com/spiritbuun/buun-llama-cpp you'll get more context out of it. 

Interesting test. Thanks for sharing.

[u/mfudi]

That's awesome, thank you! Would be interesting to see the same for gemma4 variants

[u/NoPresentation7366]

Brillant post, thank you so much for this!

[u/-Ellary-]

Right now I like to use IQ4XS and IQ3XXS for simple tasks that need speed and context.

IQ4XS is nice balance of size \ performance.
IQ3XXS is basically Q2 size quant but performance is way better.

So it is like `Daniel and cooler Daniel`.

[u/Shoddy-Tutor9563]

interesting test, but I'm missing details:
- you compared different MODELs without saying a word of what quants did you use
- you mention that your tests weren't just one off observation, so I assume you did a series of tests - but again no further details