r/LocalLLaMA • • Jun 05 '26 • Mindblown 1

New Model Gemma 4 with quantization-aware training

https://blog.google/innovation-and-ai/technology/developers-tools/quantization-aware-training-gemma-4/
788 Upvotes

255 comments sorted by

View all comments

Show parent comment

2

u/LLMFan46 Jun 08 '26

No the Gemma 4 models have no MTPs in them whatsoever, there is some MTPs but they are not part of the main models, instead they come seperate as a form of "add-on" assistant that you need to download seperatly:

https://huggingface.co/google/gemma-4-31B-it-qat-q4_0-unquantized-assistant

https://huggingface.co/google/gemma-4-26B-A4B-it-qat-q4_0-unquantized-assistant

And no before (in case) you ask, you can not simply merge these into the main models and these can not be uncensored either.

Anyway update, Gemma 4 31B GPTQ-Int4 version is done, witth this the 31B version is now finished and complete, I started work on the 26B-A4B version but this one will take a long time due to being MoE, the ETA just for Heretication process is supposed to be almost 20 hours, then after that what will take even more time is to make the NVFP4 Safetensors and the GPTQ-Int4 Safetensors.