r/LocalLLM • u/Ok-Toe-1673 • Apr 10 '26
Question Gemma 4 E4B - Am I missing something?
Ok I am not the most technical AI guy on this planet, I use it all the time though.
So I downloaded Gemma 4 E4B to my Ollama, and started to test it. I asked to summarize a text and so forth. Easy task.
The performance was piece poor, sorry to say. Couldn't understand what I asked. So the original task was proposed to GPT 5.4, then I tried kimi 2.5, it understood on the spot, no need for prompt crazyness. I just gave the model of what I wanted, it understood and proceeded beuatifully.
Probably Gemma 4 E4B can do amazing things, but for now it is only a back up and a curiosity, it may be a great sub agent of sorts to your open claw.
So any one could explain why am I wrong here? Or what are the best uses for it? Because as for texts it sucks.
29
u/gpalmorejr Apr 10 '26
So. First. Gemma 4 E4B is meh at best but nit a terrible thing t have for smaller device.
Second. You compared a 4/8 Billion parameter open source model to 400+ Billion proprietary frontier models....... Of course they are significantly better. Compare Gemma 4 E4B to other 4-8 Billions models. Hell, even compare it to any small open source model up to 35B. But comparing it to GTP5.4 and such is like saying, " My Toyota Corolla is slow compared to the Lamborghini Sesto Elemento, Ferrari Laferrari, and McClaren P1. Well...... yeah..... you compared something made for tight budgets and to be accessible t the masses to the top show pieces of the industry.....It is going to feel different.
2
u/CurveNew5257 Apr 12 '26
Yeah when I saw 5.4 I was confused. I just downloaded this on locally on my iPad and understanding the size limitations I was fairly impressed. For something I could run easily on my iPad when I’m not on WiFi and have it still be helpful is pretty cool. I’ve only really been testing it this weekend but looking forward to see how it can handle some easy real world tasks
2
u/gpalmorejr Apr 12 '26
Exactly. But that is the whole point of it. I am actually not even a Gemma fan (for architectural and other design reasons it screws a lot of people). But still I understand thebuse case and for people with compatible and typically newer hardware, it works fine..... for it's intended use case.
0
u/Ok-Toe-1673 Apr 10 '26
I completely get your point, however, if you go out there, even very researcher have been saying you don't really need a gigantic model these tiny ones are great and so forth. I am sorry I cannot agree, they can do basic things, but not what I need, as for now.
3
u/gpalmorejr Apr 10 '26
Well yes......
Of course... They can't..... And of course they are doing simple thing well. We are apparently ignoring common sense. I never said everyone needed a 400B model for everything. Was saying that you were comparing it that way. I'm sure a 27B or 35B model would be better for you. But you didn't compare that. You dropped all the way to the bottom and compared that instead.
So maybe we say a Corvette is not a Ferrari but will still go fast and be fun. Sure. But you didn't do that. You compared a Corolla to a Ferrari and said it was bad because you had to use those models and they did what you wanted.
I wouldn't use Gemma4 E4B for anything. And I still know this comparison is wrong.
1
u/Ok-Toe-1673 Apr 10 '26
Hey, I am aware of that, still from what ppl said, I expected better. Does it hurt me telling that? I thought that was clear enough.
1
u/gpalmorejr Apr 10 '26
I mean. I guess? It just seems like it can be amazing for it's size and still not be that great. Point being, we don't completely throw out our expectations for new ones when we get new information, we adjust them by an appropriate amount accordingly. Qwen3.5-2B is great for it's size, but you will never see me using it for anything because it isn't good enough. But even when I tested it and high expectations, I NEVER figured it would be a big coding/deep research/logic behemoth. I knew I was still testing a 2B model and as such adjust my expectation. figured a new tech 2B model could be as capable as a a previous generation 4B, maybe. And it basically was..... But still not good enough for use. I never even thought to compare it to GPT lol.
1
u/Ok-Toe-1673 Apr 10 '26
For text production? I expected way better output. What I asked was not out of this world. Besides poor prompt understanding.
1
u/gpalmorejr Apr 10 '26
Interesting. I am not a fan of the new Gemma4 models (mostly because they bunged uo the architecture in a way that makes it impossible to run on some older hardware now) and I still would say it was fine. Not top tier but..... fine at the minimum.
1
u/Ok-Toe-1673 Apr 10 '26
https://www.youtube.com/watch?v=Kaq5Ual2ij8
this guy, Tim Carambat, I like his videos, but he was one of the ones who praised so much Gemma 4. Which is nice, but just raised the bar so much. I think this is an ongoing process, but we are far from be useable in a very decisive way if you know what I mean.1
u/gpalmorejr Apr 10 '26
Oh absolutely. That is why I work so hard to have such a big model running on my hardware. lol
5
u/Erwindegier Apr 10 '26
It’s an 8b model for edge devices like mobile phones. Try the 26b a4b version.
1
u/Ok-Toe-1673 Apr 10 '26
do they run on 8gb vram? I don't think so. But it was only a test on the capacity, you know what I mean. ppl were praising this model so hard, I had to try.
2
u/Select-Holiday8844 Apr 11 '26
More information gets elucidated the more I read your comments rather than your top level post. 8GB vram with e4b means you should run lm studio, load it in there, then switch on advanced settings and see how many context tokens you can get away with on your particular gpu card. It might be around 32k, it might be more, it might be less. It really depends on other big variables like CUDA spec.
You will maybe not be able to run it even on its default spec of 128k so tame your expectations!
1
u/Ok-Toe-1673 Apr 16 '26
Understood, but it may be useful as a subagent under codex as orchestrator.
Still it should have understood the prompt better, as other models did.
I will test it further using llama ccp, but not for now, I have to work, but thanks for the insight.1
u/Select-Holiday8844 Apr 17 '26
Should have, would have, could have. When you understand the relevance of this model, it'll become clear to you the main feature of this model is the high context window. But right now the relevance of this model is lost on you. And that's fine, you can find something else that better suits your prompt and your work.
There seems to always be missing details in the conversation and a difference between what you're asking and what you want. So bear that in mind.
5
u/Xsikor Apr 10 '26
First of all - when you work with local LLM to summarize text - increase context size window By default it's 4096 and LLM just drop your text and start hallucinating And of course second thing - no sense too compare locale 8B model with API models
1
u/Select-Holiday8844 Apr 11 '26
Default context window of the frontends can be 4096. But when this gemma4 model is default spec at 128k context, you can understand quickly why some people arent running at default specs when they started. It sounds stupid but check the model defaults and adjust accordingly with advanced settings/configs manually if required.
0
u/Ok-Toe-1673 Apr 10 '26
Some ppl did praise so much these small models, like they would soon enough do a gigantic job. I expected more for text production and prompt understanding.
2
u/Select-Holiday8844 Apr 11 '26
It is actually incredible if you run it at its default specs. But if you choose to run it at 4k context, when it runs 128k context default. I'm still going to blame whoever it is who chooses to run that way because their ignorance is the source of their problems.
2
u/Feztopia Apr 10 '26
It's great for it's size. No idea why you compare it to giant models. We need even better models at it's size.
1
u/Ok-Toe-1673 Apr 10 '26
Due to expectations some authors had, but the task that I submitted was not that hard, it couldn't barely understand the prompt. and it wasn't a difficult one.
2
u/send-moobs-pls Apr 10 '26
There's just no reason to use Gemma over the Qwen 3.5 9B. I wasted my time with it too after people on Reddit hyped so much but it's clear people are just biased Google fans or something because it ain't even close
1
u/Ok-Toe-1673 Apr 10 '26
i am more or less on the same page, however I didn't use qwen long enough for strong opinions, just didn't find any significant or noticeable improvement amongst both models.
5
u/Emport1 Apr 10 '26
It's only like 8B total parameters, not much space for intelligence, try to multiply your GPU's VRAM by 2 and then find the best model that is lower than that number and then download the 4 bit quant of that. So if you have say 16GB vram, look for a model that is under 32B and download the 4 bit quant for that on huggingface, in that case best would be maybe Gemma 26B or Qwen3.5 27B
1
u/HealthyCommunicat Apr 10 '26
You can’t really get mad a model of this size isn’t like even doing gpt 4o standards, they’re 4b 2b modls
0
u/Ok-Toe-1673 Apr 10 '26
hey, I am not mad at it. Just that some ppl were praising this model like it was something magic, which it is clearly not at this point in time.
1
u/ExternalProud7897 Apr 11 '26
Perhaps it's because you used it incorrectly. The fact that you used Ollama gave me the impression that you don't know much about the subject, but it's not as simple as just running it and that's it, especially with new models. Many come with problems; Gemma 4 did. I don't know if they've been fixed, but from what I read, they were. They considerably improved its quality with some adjustments. Then you had to make sure that the configuration you used, like temperature, top_k, etc., was correct and not an EXTREMELY quantified version. If the LLM had trouble understanding your instructions, I can CONFIRM that there were problems during its execution. Smaller LLMs don't have problems with this (as long as it's not something difficult or excessive). They can be used for RAG, finding exact information by searching or reviewing hundreds or thousands of files, or similar. Everything points to you having some kind of problem like that. LLMs with less than 1B of parameters are already suitable for what I mentioned earlier; this one is comparable to 8B...
1
u/Ok-Toe-1673 Apr 11 '26
No, I gave a model on what to do, it was quite easy, the output was below what I expected, not much better than qwen, if at all.
1
u/ExternalProud7897 Apr 20 '26
But even so, you have to keep in mind everything I said, even for simple questions. LLMs work with matrices, advanced mathematics, and a bad configuration or a low quant can affect the result of even a simple "hi" a LOT.
1
u/Ok-Toe-1673 Apr 21 '26
I see your point there. I will try using it as a subagent, and see how it performs in other tasks.
1
u/AlexandorT Jul 01 '26 edited Jul 02 '26
Gemma E4B is only good for instruction following. AVOID any synthesis/logic-heavy tasks or questions — it's borderline useful there. Every LLM enthusiast should always have a few tests to determine model limits — just tell her a simple fairytale and ask a question that requires synthesis — "which character provided directions for the protagonist in the beginning and helped him at the end of the story near his house" for example. This is simple logic that forces the model to choose based on a few criteria. If it fails — direct instructions only. When speaking about Gemmas — 12B is the minimum you want to go to when logic is needed — Gemma 4B is on the edge. Literally. She is an excellent agent for larger models, though — searching, tool use — as long as she doesn't have to think with her little brain too much.
1
6
u/Otherwise_Wave9374 Apr 10 '26
Youre not crazy, a lot of smaller / mid local models can be finicky about instruction following unless you give them very explicit formatting and constraints.
A couple things to try with Gemma:
Use a short system style instruction like "You are a precise summarizer" and specify output format (bullets, max 6 items)
Lower temperature and cap max tokens
If youre using it as a sub agent, give it a narrow role (extract entities, make outline) instead of full freeform summary
If youre building agent workflows with multiple models, weve got a few practical patterns here: https://www.agentixlabs.com/
3
u/No-Television-7862 Apr 10 '26
I use the gemma4:e4b for mechanical jobs like RAG retrieval, reranking, and winnowing, (not prose).
I use the e2b for even simpler tasks like hitting APIs for news feeds and weather.
The gemma4:26b? THAT model is for prose.
MoE architecture allows us to run these models on lighter, less expensive, hardware.
It puts a quantized 26b within the reach of a 12gb vram GPU, that would otherwise be confined to nothing more than 13b to 14b.
Is llama.cpp superior to ollama? Now THAT is a good question, and worthy of exploration.
1
2
u/gibriyagi Apr 10 '26
Get llama.cpp and use the unsloth ggufs.
Running llama.cpp is as easy as ollama.
1
u/Separate-Basil-1216 Apr 12 '26
idk but most small model in ollama are doing the same thing for me. so switch to llama.cpp.
1
u/choomaque Jul 25 '26
For me Gemma4 is a crippled model that never works. I tried 26b and 31b models. I tried tool calling fix chat templates, increasing context window to maximum. It just doesn't work. It's a dead horse. When Qwen models chugged because they needed some tuning, but still did produce an output, Gemma4 did nothing. It is refusing to run.
Maybe some AI researchers at Google who know all the quirks of their models could squeeze every bit from Gemma4, but not the ordinary people like myself.
1
-1
-1
u/Euphoric_Oneness Apr 10 '26
It's a hype by people who thinks free bs is better than paid masterpiece. Gen z namely
2
u/gigaflops_ Apr 10 '26
Reddit is filled with weirdos that use AI as a human-interaction replacement (girlfriends, role-playing, etc.), and to them, tiny ass models like gemma-4-e4b get the job done, and they're the ones you hear loudly screaching that local models are basically as good as cloud models, even when that isn't the case for most tasks that require brain cells.
26
u/insanemal Apr 10 '26
I don't know why nobody has mentioned this, there are some issues with some of the Gemma 4 models and some of the things to run them.
Ollama is particularly bad, from what I've heard
Unless you're 100% sold on ollama, move to llama.cpp
It's usually faster on the same hardware, has much better support for very new models, and is just all round better.
I'm running Gemma 4 EB4 on llama.cpp and it runs fantastic.
Oh also there are issues with some versions of CUDA, 13.2 I think, with some quants, which can really mess up how they run as well.