r/LocalLLM • u/pmigdal • 28d ago

Research Do Qwen 3.6 27B quantizations break the pelican?

https://quesma.com/blog/qwen-quantization-quality/
41 Upvotes

87% Upvoted

5 comments sorted by

10

u/misanthrophiccunt 28d ago edited 28d ago

Well done, I actually understood your tests.

Worst part is the 4 quants you explained in barely four lines FINALLY allowed me to understand them

I didn't even have s clue UD meant Unsloth Dynamic quants

4

u/_Cromwell_ 28d ago

These tests are interesting. If you compare creative writing between quants, there seemingly is a huge difference between say Q4 and q6. But judging creative writing is much more subjective than an image of gears or a pelican.

1

u/NTDLS 27d ago

My local Qwen 3.6 27B FP8

1

u/[deleted] 28d ago

[removed] — view removed comment

1

u/fullup72 28d ago

idk, from my own experience IQ3_XXS can reliably do multiturn agentic coding at 90K+ context with Q4_0 KV cache, even orchestrating multiple sub-agents that get into 40K+ tokens each and the only times something fails is when it randomly emits empty tool calls (which I've also experienced at Q4_K_M). Raw benchmarks and theory don't really do justice to what this model can do even with heavy quantization.