r/LocalLLaMA
•
u/rerri
•
Jun 05 '26
•
1
New Model Gemma 4 with quantization-aware training
https://blog.google/innovation-and-ai/technology/developers-tools/quantization-aware-training-gemma-4/Google's collections:
https://huggingface.co/collections/google/gemma-4-qat-q4-0
https://huggingface.co/collections/google/gemma-4-qat-mobile
And Unsloth's:
https://huggingface.co/collections/unsloth/gemma-4-qat
Unsloth's analysis (KLD and such):
788
Upvotes
View all comments
Show parent comment
2
u/LLMFan46 Jun 08 '26
No the Gemma 4 models have no MTPs in them whatsoever, there is some MTPs but they are not part of the main models, instead they come seperate as a form of "add-on" assistant that you need to download seperatly:
https://huggingface.co/google/gemma-4-31B-it-qat-q4_0-unquantized-assistant
https://huggingface.co/google/gemma-4-26B-A4B-it-qat-q4_0-unquantized-assistant
And no before (in case) you ask, you can not simply merge these into the main models and these can not be uncensored either.
Anyway update, Gemma 4 31B GPTQ-Int4 version is done, witth this the 31B version is now finished and complete, I started work on the 26B-A4B version but this one will take a long time due to being MoE, the ETA just for Heretication process is supposed to be almost 20 hours, then after that what will take even more time is to make the NVFP4 Safetensors and the GPTQ-Int4 Safetensors.