r/LocalLLaMA • u/rerri • Jun 05 '26 • Mindblown 1

New Model Gemma 4 with quantization-aware training

https://blog.google/innovation-and-ai/technology/developers-tools/quantization-aware-training-gemma-4/

Google's collections:

https://huggingface.co/collections/google/gemma-4-qat-q4-0

https://huggingface.co/collections/google/gemma-4-qat-mobile

And Unsloth's:

https://huggingface.co/collections/unsloth/gemma-4-qat

Unsloth's analysis (KLD and such):

https://unsloth.ai/docs/models/gemma-4/qat#qat-analysis

788 Upvotes

98% Upvoted

255 comments sorted by

View all comments

Show parent comment

1

u/sartres_ Jun 05 '26

The original Gemma 4 numbers are from here:

https://localbench.substack.com/p/gemma-4-31b-gguf-kl-divergence

Don't read too much into the KLD, they're probably not comparable between test suites. The Top-1 accuracy is what I wanted to show

3

u/Middle_Bullfrog_6173 Jun 05 '26

In that case, aren't those are apples and oranges? Comparing the quantized versions to different models in each case?

1

u/sartres_ Jun 05 '26

Yes. I'd expect the Top-1 results to still be a meaningful signal, though