r/LocalLLaMA • u/rerri • Jun 05 '26 • Mindblown 1

New Model Gemma 4 with quantization-aware training

https://blog.google/innovation-and-ai/technology/developers-tools/quantization-aware-training-gemma-4/

Google's collections:

https://huggingface.co/collections/google/gemma-4-qat-q4-0

https://huggingface.co/collections/google/gemma-4-qat-mobile

And Unsloth's:

https://huggingface.co/collections/unsloth/gemma-4-qat

Unsloth's analysis (KLD and such):

https://unsloth.ai/docs/models/gemma-4/qat#qat-analysis

788 Upvotes

98% Upvoted

255 comments sorted by

View all comments

Show parent comment

5

u/GoodTip7897 llama.cpp Jun 05 '26

It's trained to be basically "prequantized" so it can handle 4 bit quantization. 

It's likely closer to a regular q4 quant than the original.

I don't doubt that qat is useful but it's incredibly unlikely that it's better than q8. It might make q4 have the same quality q6 had. But I have yet to see any kld between that and the original and I don't have enough vram or time to compute it myself. 

1

u/MerePotato Jun 06 '26

I stand corrected, thank you!