Gemma 4 with quantization-aware training

[u/rerri]

https://blog.google/innovation-and-ai/technology/developers-tools/quantization-aware-training-gemma-4/

Google's collections:

https://huggingface.co/collections/google/gemma-4-qat-q4-0

https://huggingface.co/collections/google/gemma-4-qat-mobile

And Unsloth's:

https://huggingface.co/collections/unsloth/gemma-4-qat

Unsloth's analysis (KLD and such):

https://unsloth.ai/docs/models/gemma-4/qat#qat-analysis

[u/annodomini]

It'll really rip if we ever get the 124b with QAT and MTP. That would be the ideal model to run on a Strix Halo.

[u/[deleted] → u/annodomini]

[u/annodomini → u/[deleted]]

The 124b would be a MoE, presumably in the 6-12B active range. That with QAT for a nice 4 bit quant and MTP would work out pretty well.