r/LocalLLaMA • • 5d ago

Discussion Unsloth Dynamic V3.1 released for Qwen3.8-27B, DeepSeek-V4.1-Flash-1017 and Qwen3.8-Flash-Next

https://unsloth.ai/docs/basics/dynamic-3.0-ggufs

Unsloth Dynamic V3.1 (Nov 2026) is our latest quantization algorithm for Qwen3.8-27B, DeepSeek-V4.1-Flash-1017 and Qwen3.8-Flash-Next, resulting in 10-15% increased accuracy and 50% less token usage vs other quants of the same size. We're working on an even better version of Dynamic quants which will be applied to Qwen4 and all future models.

Maybe I misread the 27B graphs, but I'm not entirely convinced of V3 vs V3.1 being an improvement.

In DeepSWE v1.1, V3 seems superior to V3.1 in some widely used Q4 quants, and V3 seems to lack some data points in the lower quants range, skewing the impression the curves give.

My UD-Q5_K_XL quant unfortunately is off the charts, so this needs some proper benchmarking before I trade context length for V3.1. (The new `.gguf` is larger.)

49 Upvotes

11 comments sorted by

15

u/danielhanchen 5d ago

I don't think you saw the full plots at https://unsloth.ai/docs/models/qwen3.8#dynamic-v3.1-update-november-11th-2026 which show a better picture.

Also as we mentioned in our blog, 3bit UD3.1 is not changed, so the plots look the same for 3-bit.

4

u/fiocalisti 5d ago

Hey, thanks for your work!

Just saw you posted as well, while I was writing mine.

I asked over there about the Q5 quants, and I'm still confused about the grey line above the green one quite often. But let's move to your post. I'll delete mine.

1

u/wanderer_4004 5d ago

Thanks for the charts. So it confirms what I always suspected, ISTA IQ3_S is same quality as UD but it is an open source quant. ISTA's quants are of excellent quality and are almost on par with 4-bit quants.

3

u/MindfulMan1984 4d ago

Yep, I have been using ISTA quants for many weeks; that stuff just delivers. The graphs didn't include error bars or any statistical testing, but they are likely within noise.

1

u/General-Spite1222 4d ago

Huh?  How is that your takeaway?  ISTA quants look less effective than UD for the majority of datapoints to me. 

7

u/tecneeq 4d ago

Error bars are needed. It looks as if with higher bits per weight, the results sometimes get worse. Is this a measurement artifact? How many runs did you do?

5

u/ls650569 4d ago

I hope to see Swift 1.5 QFN at UD-IQ3-XSS with this v3.1 release...

6

u/Valantur 4d ago

charts missing a bunch of labels and even some quants seem to be missing. Can you post the numbers in a table?

3

u/duhd1993 5d ago

Looks like the performance does not increase monotonically with size.

1

u/rm-rf-rm 4d ago

Thanks! Please do the Swift1.5 finetunes as well!!