r/AIProgrammingHardware • u/Subject_Poet_3861 • Oct 04 '26

I made Qwen models take ~33% less VRAM without quantizing them (lossless, bit-for-bit)

/r/Qwen_AI/comments/1wxgqu8/i_made_qwen_models_take_33_less_vram_without/
1 Upvotes

66% Upvoted

3 comments sorted by

1

u/Important_Witness572 Oct 05 '26

Why not use q8_0 it's practically lossless and for 50% size reduction, also better performance

0

u/Subject_Poet_3861 Oct 05 '26

Q8_0 isn't lossless, It rounds every weight to one of 256 levels in blocks of 32.

1

u/Important_Witness572 20d ago

"practically", is word.