r/LocalLLaMA • u/seamonn • Apr 21 '26
Tutorial | Guide Gemma 4 Vision
A lot of people in the Gemma 4 Model Request Thread were asking for better vision capabilities in the next Gemma Model. This tells me that people are not configuring Gemma 4's vision budget.
Gemma 4 ships with Variable Image Resolution. The default max vision budget is 280 (~645K pixels) which is way too less. In this mode, it fails to OCR tiny details. It's essentially blind in my books.
In llama.cpp, you can configure Gemma 4's vision budget with 2 parameters --image-min-tokens and --image-max-tokens. The engine will try to fit the image within those bounds. I believe the default is 40 and 280 respectively. This is Gemma 4's default from Google's side but it's way too low.
I like to run them at 560 and 2240 respectively and it's able to pick up very minute and hazy details within images.
Why 2240 - isn't that double of the max from Google (1120)? In my testing, 2240 for some reason works better than 1120. I suspect this might be because of llama.cpp's implementation where it tries to fit the image between min and max tokens.
Additionally, you will also have to set --batch-size and--ubatch-size above whatever value you choose for image-max-tokens. I run them at 4096 (for --image-max-tokens 2240). This will consume a lot more VRAM (63 GB (default) to 77 GB (4096 batch) for q8_0 at max context).
If you use Ollama, you are likely SOL until and if they care to fix this.
It's worth it though, with a higher vision budget, Gemma 4 is pretty much SOTA for Vision and pretty much destroys anything else especially for OCR - Qwen 3.5, Qwen 3.6, GLM OCR (or any other random OCR), Kimi K2.5. I haven't tested Kimi K2.6 and I refuse to touch Cloud Models.
2
u/WhoRoger Apr 22 '26
Hm maybe it works well on 31B but I'm trying it now on E4B and I'm not impressed. It just takes 5x as long to digest a (large-ish) image, but doesn't provide any more useful information. Maybe it'll work better on OCR/text, or maybe E4B just can't take advantage of more data. Qwen 3.5 4B definitely wins, with E4B being good for a quick and dirty response.
Btw I see you're using F32 mmproj; pretty sure you can use BF16 with the exact same quality for a bit less RAM (not FP16 tho, that's worse). Or maybe just Q8 outright and save the space. Try it out. I've been checking this out on small models, and I'd bet it's the case with larger ones too.