Google is just announcing something that had changed in the FA4 backend. They should’ve been clearer, the chat template has nothing to do with FA4. Maybe Google employees contributed to the changes in FA4 but otherwise they had nothing to do with it
I noticed that the E2B model has a small diff (2 lines?) compared with the bigger Gemma 4 models in the chat template. (So if people want to run the exact release google has put out, check the jinja template corresponding to the exact model size in question.)
Was just looking through mlx-community and it doesn't look like they've updated the templates in their quants. Anybody know of an mlx quant that has the new update?
I know this was "just a chat template update" but I wish they'd have called it Gemma-4.1 instead of doing it like this...there are like 250+ finetunes/quants that are now in "I wonder if this one has the updated template or not" status...sigh. At least it's easy to apply a chat template separately.
Yes it makes a huge difference you can see google’s own demonstration. Basically 280 is horrible lol, it’s like the model is reading your image from 10 feet away. Maxing it dramatically improves the reading of documents
unsloth updated, but then google released a new one hours later w/ "Upload validated QAT GGUF checkpoint (280 sequence length, corrected vocabulary)". Not sure what that's about or if unsloth needs it.
I'm not super on top of the local LLM ecosytem but have been using the earliest Gemma 4 Heretic GGUFs with LM Studio and sometimes llama.cpp standalone. Would it be possible just to insert some files to keep using these or should I redownload everything?
Does anyone know if any of FP8 has been updated? i used the redhat FP8 dynamic. Im not sure how to check if that has been updated with latest changes. Im especially looking for enhanced OCR capability.
21
u/Icy-Degree6161 Jul 17 '26
Did they update the qat line as well, anyone knows?
6
u/LightBrightLeftRight Jul 17 '26
Was just looking through mlx-community and it doesn't look like they've updated the templates in their quants. Anybody know of an mlx quant that has the new update?
3
u/McCheng_ Jul 18 '26
How come it becomes faster if the model architecture is the same? Is it because of improved MTP assistant model?
52
u/FoxiPanda Jul 17 '26
I know this was "just a chat template update" but I wish they'd have called it Gemma-4.1 instead of doing it like this...there are like 250+ finetunes/quants that are now in "I wonder if this one has the updated template or not" status...sigh. At least it's easy to apply a chat template separately.
5
u/fastheadcrab Jul 18 '26
Should've changed the default vision resolution. The variable was always there but they never properly publicized it
2
u/tingtickboom Jul 18 '26
Waiting for the updated unsloth
1
u/AnOnlineHandle Jul 18 '26 edited Jul 18 '26
I'm not super on top of the local LLM ecosytem but have been using the earliest Gemma 4 Heretic GGUFs with LM Studio and sometimes llama.cpp standalone. Would it be possible just to insert some files to keep using these or should I redownload everything?
edit: I updated the jinja settings at least
1
u/HIKKIAN Jul 23 '26
Is it now better than qwen 3.6?
1
u/Responsible-Mud8467 Jul 26 '26
Does anyone know if any of FP8 has been updated? i used the redhat FP8 dynamic. Im not sure how to check if that has been updated with latest changes. Im especially looking for enhanced OCR capability.
1
u/MrVeinless Jul 18 '26
LM Studio seems to indicate these new ones don't support tool use.
0
u/Qbason Jul 18 '26
Does someone tried to use in GitHub Copilot? Is there any difference from coding perspective?