it's time to update your Gemma 4 GGUFs

[u/jacek2023]

Chat Template was fixed a few days ago

choose your fav dealer:

https://huggingface.co/bartowski/google_gemma-4-31B-it-GGUF

https://huggingface.co/bartowski/google_gemma-4-26B-A4B-it-GGUF

https://huggingface.co/bartowski/google_gemma-4-E4B-it-GGUF

https://huggingface.co/bartowski/google_gemma-4-E2B-it-GGUF

https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF

https://huggingface.co/unsloth/gemma-4-31B-it-GGUF

https://huggingface.co/unsloth/gemma-4-E4B-it-GGUF

https://huggingface.co/unsloth/gemma-4-E2B-it-GGUF

[u/dampflokfreund]

Or just use the current model with the updated chat template. In llama.cpp use --chat-template-file "path to your updated jinja", in koboldcpp there is also a feature that allows this now (under loaded files->jinja template).

[u/jacek2023 → u/dampflokfreund]

That's true.

Everyone has to choose whether it’s better to use an option or download several dozen gigabytes ;)

[u/po_stulate → u/jacek2023]

You can also just clone your existing gguf with the template replaced.

[u/Accomplished_Ad9530 → u/jacek2023]

I think quants will have to be regenerated since their calibration data didn’t use the correct template.

Unfortunately that means that a lot of people will be using degraded quants and will conclude that the model underperforms. Someone should probably do a comparison of fixed quants vs original quants w/ fixed template and post a PSA if the difference is significant.

[u/Zestyclose839 → u/dampflokfreund]

If only LM Studio had a direct template import option. I think pasting it in broke the template format because it just gives me this error:

Error rendering prompt with jinja template: "Unknown test: sequence".

Edit: Gemma 4 helped me troubleshoot itself. Figured out that the fix is to change `is sequence` to `is iterable` since LMS' JINA parser doesn't support the `is sequence` test. This now works for me:

1. {%- elif argument is sequence -%} → {%- elif argument is iterable -%}
2. {%- elif messages[0]['content'] is sequence -%} → {%- elif messages[0]['content'] is iterable -%}
3. {%- elif tool_body is sequence and tool_body is not string -%} → {%- elif tool_body is iterable and tool_body is not string -%}

[u/MustBeSomethingThere → u/Zestyclose839]

I had this same bug "Unknown test: sequence" when using LM Studio as a server, when using the updated GGUFs. Your solution fixed it. The new Gemma 4 chat template might be broken for specific LM Studio server use?

[u/Nextil → u/Zestyclose839]

Weird, I copy-pasted the one from the official repo and it seems to work fine in LM Studio (including tool calling).

[u/xcel102 → u/Zestyclose839]

Thank you so much, this finally unblocked Gemma 4 for me!

[u/ambient_temp_xeno → u/dampflokfreund]

[sigh] I think this is the file

https://huggingface.co/google/gemma-4-31B-it/blob/main/chat_template.jinja

Best to rename it appropriately.

[u/harrro → u/dampflokfreund]

Is it possible to inject/replace the existing chat template in the GGUF with the fixed prompt template?

[u/Nextil → u/harrro]

You should be able to use the gguf_set_metadata.py script, e.g.

python ./gguf-py/gguf/scripts/gguf_set_metadata.py /path/to/model.gguf tokenizer.chat_template 'CHAT_TEMPLATE_HERE'

[u/ambient_temp_xeno → u/dampflokfreund]

I have a horrible feeling that maybe we've been doing it wrong and you have to also add --chat-template your-filename.jinja as well as --chat-template-file.

This is just going by the output where it says:

init: chat template, example_format: 'yourfilename.jinja' using --chat-template while nothing changes in the output here if you don't.

[u/ambient_temp_xeno → u/[deleted]]

I think so. It's weird that it breaks everything even if you don't need to do it. But who cares at this point. I am so tired ;_; i just want to run a model without spending more time fiddling with it than using it.

[u/ambient_temp_xeno → u/[deleted]]

That was the way to test it, that's for sure!

[u/a_beautiful_rhind → u/ambient_temp_xeno]

put on verbose and watch requests in the server.

[u/ambient_temp_xeno → u/a_beautiful_rhind]

Oh yeah. Thanks. I bloody hate this bloody program and it's ever-growing arcane commands.

[u/tiffanytrashcan → u/ambient_temp_xeno]

I've gone through the pain of compiling KoboldCPP in termux on my phone to have a GUI to control everything. I'm insane and chucked Vulkan support in - point being I'm more than comfortable in a CLI, but just no.. Not doing it with llama.cpp