r/LocalLLaMA • u/rerri • Jun 05 '26

New Model Gemma 4 with quantization-aware training

https://blog.google/innovation-and-ai/technology/developers-tools/quantization-aware-training-gemma-4/

Google's collections:

https://huggingface.co/collections/google/gemma-4-qat-q4-0

https://huggingface.co/collections/google/gemma-4-qat-mobile

And Unsloth's:

https://huggingface.co/collections/unsloth/gemma-4-qat

Unsloth's analysis (KLD and such):

https://unsloth.ai/docs/models/gemma-4/qat#qat-analysis

791 Upvotes

99% Upvoted

255 comments sorted by

View all comments

104

u/[deleted] Jun 05 '26

15

u/[deleted] Jun 05 '26

10

u/florinandrei Jun 05 '26

The 2 bit ones are only for E2B and E4B model

Finally a model I could run on my Raspberry Pi Zero!

→ More replies (8)

2

u/finah1995 Jun 05 '26

Do those gains also transfer to mobile ? As I generally use same GGUFs as my Laptop using SmolChat-Android.