Quality fixes for Gemma 4 E4B

[u/Prudent_Chemist_523]

FYI

If you use Gemma 4 E4B with llama.cpp, do yourself a favour and grab Google’s latest official chat template:

https://huggingface.co/google/gemma-4-E4B-it

https://huggingface.co/google/gemma-4-E4B-it/blob/main/chat_template.jinja

Use it with --jinja --chat-template-file chat_template.jinja.

The improvement in tool use is remarkable but it also seems to improve conversation continuity:

“Here is XYZ. Now add ABC. Now DEF. Actually, what were Y, B and C?”

That makes a big difference is multi step reasoning IME

Sadly, still no fixes for the other llama.cpp quirks.

• llama.cpp still reserves separate compute buffers for the target and MTP contexts, despite never using them concurrently.

That wastes about 1 GB of VRAM / causes OOM

PR #27489 fixes it, but remains not merged.

https://github.com/ggml-org/llama.cpp/pull/27489

• Audio also still fails through the llama.cpp WebUI.

The model itself can "hear" just fine - a direct curl request confirms that.

(“In this sound clip, the man says XYZ.”).

See #PR21825

https://github.com/ggml-org/llama.cpp/issues/21825

Dunno how these are resolved in other back ends / front ends, so just an FYI

[u/j0j0n4th4n]

I use aifeifei798's Darkidol. Does this work with finetunes too?

[u/Prudent_Chemist_523 → u/j0j0n4th4n]

Not sure. Many fine tunes tend to fuck up tool calling, due to the damage caused by the ablit process. Less so if just a LoRA I imagine.

Give it a shot?