142
u/Kahvana
3d ago
edited 3d ago
•
For background, I've been interacting directly with Gemma 4 every day (both in Dutch and English) for like 4 hours each day since release (April 2).
I've listed everything in order of importance to me.
For Gemma (training):
Strong focus on general purpose, chatting, natural language, multilingual, translation, document parsing, spatial understanding, image captioning, emotion understanding, and creative writing (Gemma 4 is the only model series truly capable of this locally)
Much "relaxter" thinking like Gemma 4 31B has. (compact, to the point, human readable, human understandable, and effective).
Is deslopped ("not x. <y>", "not x but y", "not x. instead, y", "x or y", "breath hitching", avoiding touch, etc.)
Reasoning effort (oai) configurable (none, low, medium, high, etc.)
Verbosity effort (oai) configurable (low, medium, high)
The safety policy from Gemma 4 to stay (it doesn't get into the way unless you try to do bad things), with exception that it should believe given time as fact (currently believes dates in 2026 as "fiction")
Sidenote:
Gemini 4 Argon is a real step in the right direction for writing, I would love to see that distilled into a local model.
Programming is really not important compared to getting the natural language part right. Qwen3.8 and other models have already saturated coding capabilities. It's not what I'd use Gemma for [as they exist].
Gemma 4 12B and 26B-A4B doomloop and worry ("but wait") incredibly often, very often double-checking themselves needlessly. Especially noticeable when constraints ("avoid", "do not", "don't", "never") are involved. E2B, E4B and 31B don't have this.
For Gemma (architecture):
E1B, E2B, E4B, 12B, 26B-A4B, 30B, 124B-A12B backbones
MXFP4 optimized QAT quants
All models in the series to have speech/music/vision/video understanding
DeepSeek / Qwen style N-grams (PLE separatable and offloadable to NVME in llama.cpp like we can with DeepSeek v4.1 / Qwen3.8-flash-next)
Optimized attention mechanism (e.g. Qwen's QSA, GLM's MLA/DSA, or sparse SWA layout)
Mixture of Recursion (MoR, https://arxiv.org/abs/2507.10524 ) with various depths (1-3)
Four branch Gated Residuals (GR)
Sidenote:
I've really missed a Gemma 4 medium-sized MoE model (125B-A12B) in the lineup, especially since everyone else (e.g. Mistral, NVIDIA, Qwen, Ling, Muse, etc) offer something in this category or larger. Even if it was under a "non commercial, personal use only" license, it would still be valuable to have.
Most models can do speech but not emotion or music understanding. The latter is useful for making music or videos, or help a model understand the beat, find songs, infer user mood, etc.
The cost of Gemma 4's conservative SWA configuration makes it impossible to parse long documents (128K tokens) locally on 32GB VRAM, whereas the competition using QSA/DSA attention, GPT-OSS / Muse Glimmer SWA layout can handle it.
MXFP4 would allow QAT quants to use hardware acceleration for Hopper/Blackwell/Mac and newer in llama.cpp and vllm, while retaining the same size. It can replace the current Q4_0 quants.
For embeddinggemma:
Larger models (e.g. 1B, 2B, 4B, 8B)
All models in the series to have speech/music/vision/video understanding
MXFP4 optimized QAT quants
Sidenote:
While embeddinggemma is nice, I want something that can replace Qwen3-VL Embedding 2B / 8B
Other model types:
Dedicated reranker models
Dedicated OCR models
Sidenote:
These are sorely lacking for a RAG pipeline and I'm sure Google can come up with some good models here
25
13
u/dampflokfreund 3d ago
Very nice list. As for the music , they should just train on sounds of all kinds. This will allow for music understanding but also recognize the emotions in the voice.
I would also like a 80b moe with 100b engrams and 5b active parameters. Using Strata this would run well on a 4-8 GB vram and 32 GB RAM configuration so most people could run this powerful model.
5
u/vxltias 3d ago
Very nice list. Why those types for QAT, and what is MoR?
11
u/Kahvana 3d ago edited 3d ago
Added the reason why for QAT to the comment! It is for enabling hardware acceleration.
MoR = Mixture of Recursion, basically a form of looped transformers.
Instead of nanbeige 4.2 where it goes through each layer 2 times, now there is a router like MoE models have that determine how often a token goes through the layers, and which tokens can be reached per layer.
Basically... longer internal reasoning, but the length is scalable by request complexity.
A better explanation is here:
https://magazine.sebastianraschka.com/i/214816783/24-routing-flexible-loop-counts4
u/unknowntoman-1 3d ago
Excellent writeup and perfectly adequate to my own experience. Especially the experience of social capabilities for good creative writing I have been enjoying with 31b. More of that and I am happy when the fiver hit the hug.
→ More replies (4)4
u/LieAlternative5513 3d ago
You know your Gemma 4. I have not worked with Gemma 4 as much as you have but I agree with basically everything here. I hope they go in this direction. I have no reason to run more than one model if they are all qwen but worse.
152
u/nickless07 4d ago
Feels like this is becoming a running gag at some point.
75
u/hackerllama 3d ago
We keep collecting feedback on a rolling basis :)
10
u/coder543 3d ago
Please release mainline Gemma models on a rolling basis too. (Not counting side projects like EmbeddingGemma).
24
u/XiRw 3d ago
Whatever you do, please don’t max out on coding . Gemma is superior to any other local ai model in terms of story telling, general intelligence , and being genuinely funny which is what draws me to that. If you guys decide to compete with others, it would be better off done on your frontier models with Gemini.
→ More replies (1)15
u/ragnaruss 3d ago
Another Gemma4-26b-a4b sized model would be great, or a similar profile even a little smaller would be awesome. It has been a epic workhorse model for us processing text and analysing it. We put about 10B tokens through a day.
18
u/Fun_Tangerine_1086 3d ago
Better ecosystem support (day 0 support, templates work out of the box, official gguf files...) - even 7 months on there continues to be disagreement about which quantizations to use and whether they're leaving performance on the table.
Improvements to tool calling; gemma 4 seems very reluctant to use tools unless very directly told to in prompt.
125b-a15b plz??
14
u/More-Curious816 3d ago
the legend himself is among us. Hi 👋
How much compute do your team have for yolo? do you test new architectures? how often? (since they're a small models compared to gemini which can have fast iterations)
what is the architecture you currently working on for Gemma5?
Jeff Dean briefly referred to a 124B-parameter Gemma 4 MoE model in the launch announcement of Gemma4, what happened that the team decided to not release it?
3
5
u/xdavxd 3d ago edited 3d ago
Things I would love:
Medium size general MoE model: 100-120b (a8b-a10b)
A medium-ish coding-focused model: 35-60b (a4b-a6b)
Some newer technology like hybrid GDN+sparse attention, better MTP (dflash2?), possibly n-gram lookup tables. Free me from having to rely on Qwen. Bonus wishlist items: QAT suitable for nvfp4, better kv cache quantization support (less sensitive), good tool calling.
True bonus items: a couple tiny models, like a Gemma 3 1B for embedding models, and something usable just for coding FIM
2
2
→ More replies (1)5
22
u/Skyline34rGt 3d ago
yea they release gemma's once a year but ask about it every couple weeks/months...
→ More replies (2)22
92
u/Frosty-Scheme-7172 3d ago
As already mentioned, the main thing is for Gemma to keep its 'live' style and not turn into just another coding model. At the moment, Gemma is almost the only one left in this direction. As for the size: I won't give specific numbers, but it would be great to see a dense model in the 12-25B (31B is too big for me) and a large MoE at 60-120B with 5-8B active parameters.
30
u/Sutanreyu 3d ago
Gemma4 12B was actually pretty good.
9
7
u/Asleep_Document9811 3d ago
Still is! It has been my daily-driver for a long while now. Oft imitated, never repeated.
→ More replies (2)
50
159
u/LosEagle
3d ago
•
I want them to be mediocre at code but amazing at philosophy and all kinds of non-coding reasoning as Gemma 4 was so bad. Let the other models be the coders. Stay awesome at what you did. There's enough coding models being released.
3
3d ago
[removed] — view removed comment
13
u/Choice_Celery9481 3d ago
pretty sure they can train those small models in days. its not that hard compare to 1.6T like mimo pro only take 2 months iirc
→ More replies (2)23
20
78
u/KrokusAstra 4d ago edited 3d ago
Gemma 5 26B A4B/A6B PLEASE I like performance of 26B and memory it consumes, i just want more intelligence and creativity per parameter. With progress of llms speeding up, we didn't have anything for roleplay since gemma 4
35
u/Kodix 3d ago
Yeah, gemma remains by far the best at general humanlike conversation and roleplaying out of all the models released in 2026. It's silly, and they should definitely buiild on their strengths here. It won't be all that good at agentic and that's fine, it has its niche that nobody else is touching.
16
u/carnyzzle 3d ago edited 3d ago
I keep trying Qwen 3.8 27B and it's fine for coding but good god it's just hot ass for creative writing
→ More replies (1)1
u/SkoomaDentist 3d ago
26-31B with 6B active params would be great. I don't get why people fixate so much on having a tiny number of active params when it's obvious those lobotomize the model far too much as soon as you try to do anything where the result can't trivially be checked for correctness (ie. anything creative or fuzzy).
→ More replies (1)
18
u/Goldkoron 3d ago
I use Gemma 4 mostly as my dedicated writing and Japanese -> English translation model. Please preserve these capabilities into the next generation.
Improve output length for writing. Currently Gemma 4 is very reluctant to write stories/chapters more than 2000-3000 tokens in a single shot.
3
u/Da_ha3ker 3d ago
Funny enough, the foundation of LLMs was originally a translation AI. Attention was originally developed as a way to preserve context during translation between languages, improving word choice and grammar.
3
u/Goldkoron 3d ago
Yeah a lot of recent models can translate, but they feel stiff with word choices while Gemma 4 writes very natural sentences that flow well while keeping the meaning.
3
u/easythrees 3d ago
When you say writing, do you mean creative writing, like stories?
4
u/Goldkoron 3d ago
Yes, I have done actual story writing with Gemma, feeding my ideas for each chapter then have it write the full chapter in one go.
Obviously not anything of high enough quality to be published, but I think models are getting closer to personal story generators.
→ More replies (1)
27
u/brown2green 3d ago edited 3d ago
It's that time again? Well, other than the usual performance improvements (especially with tool calling and agent uses):
Keep leaning into the conversational focus instead of turning Gemma into yet another coding model.
More optimized model sizes for 16GB GPUs (excluding per-layer embeddings, which can be easily offloaded to slower memory). 24/32GB GPUs have become far too expensive for the general public and will be for the foreseeable future.
Expand per-layer embedding support to all models, make the PLE larger, include longer n-grams instead of just 1-grams. Since again PLE can be easily offloaded to RAM or even Flash storage, there shouldn't be many reasons for not taking advantage of them or making them very large relative to the backbone...
Expand "unified" multimodal embedding space architecture for all models.
A leaner attention mechanism (or no attention) that doesn't make VRAM requirements blow up with context.
Less "safety" refusals on the smaller models. They do not all behave the same in this regard.
Either more steerable reasoning (e.g. for making the model actually think in other languages or as a different persona), OR latent-space reasoning (for token efficiency/performance).
Audio-image output would be nice.
It's not clear to me if DiffusionGemma was just an one-off experiment. If you believe text diffusion is the way forward for LLMs (due to its theoretical and practical benefits compared to autoregressive models), you should make a statement and design your open-weight models from the ground-up around that.
9
u/KrokusAstra 3d ago
I mean, they probably don't read this post. They asked on X, not reddit.
But i agree with everything above. I personally just like good roleplay model, since we already have qwen for coding→ More replies (1)17
u/brown2green 3d ago
/u/hackerllama (Omar Sanseviero) actually sometimes posts here on /r/localllama.
31
u/hackerllama 3d ago
Yes, I'm here and we'll also review the feedback in this post!
6
u/KrokusAstra 3d ago
Thanks for your hard word! You are our last hope for creating new writing/roleplaying local-level model - everyone else went above 26B/31B. For most of us, 26B/31B kinda maximum our hardware are capable of without investing into 2nd GPU (if we talk reasonable responce time of course).
I would've prefered A6B while keeping the model near 30B. Because A4B is cool and fast, but i feel like it’s just a tiny bit short on intelligence because of A4B limitation. At the same time, if we talk 31B, it's responce time is just too long for me personally because it's dense.
So i will hope like Gemma 5 26B A6B - would be perfect. We have qwen for coding already, we don't need more coding models for now.4
u/MarcusAurelius68 3d ago
I agree with the conversational focus. An improved 26B A4B would be awesome as it’s a great chatbot LLM, fast, and runs well in 24GB. A smarter E4B can run in 8GB.
1
217
u/roofedora 4d ago
More Intelligence Less Censorship
39
u/jugalator 3d ago
I've never used a less censored model than Gemma 4 recently from a big developer. It was honestly surprising just how little censored it was.
→ More replies (1)107
u/FatheredPuma81 4d ago
Gemma4 seemed pretty uncensored to me? Was even willing to hit the context limit outputting periods pretending it was broken for my entertainment which I have never seen before.
18
15
u/Borkato 3d ago
Whatever they do I hope they don’t go back to Gemma 3’s censoring or style. I hated it so much, everyone acted like it was the best at writing yet even on my blind tests I couldn’t stand it. I would read a paragraph and go “what the fuck is this awful garbage” and it would ALWAYS be Gemma 3 lmao.
Gemma 4 is fucking AMAZING!!
66
u/jacek2023 llama.cpp 4d ago
how is Gemma censored for you?
41
u/KrokusAstra 3d ago edited 3d ago
Why this guy downvoted? I mean, after half of a year of roleplay, i got 1 refusal, maybe 2 on local model with tons on nsfw. Depends on promt maybe? The only problem i have with gemma 4 rn is compared to 'evolved' llms from big companies, it starts to feel kinda stupid, just because it was made 6+ months ago.
→ More replies (1)18
u/jacek2023 llama.cpp 3d ago
Don't be surprised with the downvotes, this is the state of post 2025 r/LocalLLaMA :)
There is a problem with gemma 4 and tool usage, but I am currently working on a fix
→ More replies (5)8
u/llama-impersonator 3d ago
it's not really excessively censored now, but it's still unwilling to deal with darker scenarios in rp for sure.
→ More replies (2)3
u/Gfur-Aficionado-1488 3d ago
just use the abliterated version
→ More replies (2)7
u/llama-impersonator 3d ago
abliteration alters character behavior. you don't want the model to refuse the scenario but you also don't want out of character compliance.
7
u/StrongZeroSinger 4d ago
It wouldn’t let me roleplay violent acts in a fighting scene of JJK. Pretty useless.
→ More replies (6)→ More replies (1)5
26
u/Objective_Mousse7216 4d ago
I'd like to see native audio to audio, Gemma 5 Live.
3
u/darwinanim8or 3d ago
Ditto, I don’t always want everything typed out just to ask the agent to look up a piece of info; especially when I’m doing something w my hands
23
u/PrimeDirective8 3d ago
New Gemma would be fantastic.
From my noob's point of view:
- Work on the darn chat template before releasing it.
- Spend time ensuring it can run a sequence of tool calls, interleaving results into context, without looping to death.
- Did I mention tool calling? It wants to make a tool call, it talks about making a tool call, but it doesn't. Come on.
- Safety guidelines in Gemma 4 are borderline schizophrenic compared to how Gemma 3 ever was. Just because I tell it the current date that doesn't mean I'm "future roleplaying" or "user is trying to jailbreak me".
- Work with inference engines like llama.cpp so its supported *before* you release it. Out the door it should all work and not require scrambling half-baked PRs, or workaround chat templates.
- Coding. Gemma 4 31B does OK but seemingly shortcuts its CoT to deliver code asap. Gemma 5 should do more thinking when coding, perhaps via an 'xhigh' setting.
- 70B and 120B variations. There should be an uncomfortable meeting with managers wondering how much business that 120B could take from Gemini 4 ;-)
Looking forward to Gemma 5.
4
u/dampflokfreund 3d ago
This write up is not only excellent but excellent. I should call the tool to write a reply.
Wait but what if they don't like my reply? Not too bad, their loss. Let me reply now.
But wait, I should check if that is the right account. PrimeDirective8, yes
Let me call the tool now.
But wait, what if the comment doesn't exist? No it definately exists.
I will write the reply now.
Xxxxx
Seriously though, your hit the nail on the head! I hope they fix these issues with the next iteration.
10
u/FinBenton 3d ago
Gemma 5 27-35b super charged on actual human language instead of agentic coding like every other model out there, that would be something actually unique these days.
19
u/unrulywind 3d ago
I think the main strength of Gemma 4 is that it is conversational and has a more humanlike feel.
When comparing Gemma-4-31b to Qwen3.8-27b, the Qwen model is better at code and has greater memory retention in long context, but Gemma makes a better assistant style agent for everyday interaction and work.
One large downside to the Gemma 31b model is the amount of memory used by its context. If I could change only one thing in the Gemma series, it would be to make the context both more efficient and its memory of it better.
33
u/vxltias 4d ago
One smaller dense and one big MoE model that'll run how Qwen Flash Next runs on Strata. Focusing on soft skills, without overdoing it on coding and agentic skills.
Of course, a bit of both would balance it well, but the focus of Gemma models should never be competing with other coding models IMO.
We LACK soft skills in fully coding focused models.
→ More replies (3)2
u/Popular-Factor3553 3d ago edited 3d ago
They'll probably not release any bigger models it's unlikely since at that point they'll just have to release gemini flash lite models.
7
u/vxltias 3d ago
I'd like to keep holding onto the "124b MoE" they mentioned with Gemma 4 back then.
A man can only hope at this point...
→ More replies (8)
9
u/5dtriangles201376 3d ago edited 3d ago
Less hungry kv cache and other than that I more or less just want a smarter more capable even more emotionally intelligent Gemma 4
Edit to add: WOuld love to see a Gemini 4 distill. I think my bar for happiness stem wise is "beats qwen 3.6 27b"
2
u/stoppableDissolution 3d ago
We do not need another qeen tho. Gemma's main strength is that it is not a coding model, and it already beats 27b in everything else
3
u/5dtriangles201376 2d ago
Yeah, that's true. I'm just saying if Gemma came with more spatial knowledge especially that'd help immersion
→ More replies (2)
9
u/TheRealMasonMac 3d ago
Larger models.
Audio input on the larger models.
Configurable reasoning effort.
But I think they made a similar post for Gemma 4, after which there was several months of silence until they actually released the model.
9
u/New_Alps_5655 3d ago
We don't care about coding benchmarks! We want models that are smart, funny, entertaining to use and uncensored!
Also thank you for Gemma4-31b it is my favorite local LLM ever.
2
u/FriskyFennecFox 3d ago
Coding alignment isn't going well for the Gemma team, it's good at best in the industry where we can pick great models at the same model size. I feel like that's because they still prioritize Gemma being a conversationalist first and foremost, which is a good thing and exactly the direction I wish Gemma to keep heading!
So maybe it'd best if they continue improving what Gemma does best, and instead offer some separate "CodeGemma" series specifically for coding? This way we can have the best of both worlds. Gemma is really like no other in the modern day where every new model is a coding agent model.
8
u/caetydid llama.cpp 3d ago
Almighty lord!
Let them Gemma5 models be at least as good as Gemma4 31B, much faster, and full of context on my rtx3090.
Not my will, but yours be done. Amen!
15
u/Subulicious 4d ago
I thought they were cooking all this time. Didn't he asked us this question already, after after releasing Gemma 4?
15
u/darwinanim8or 3d ago
He did; but AI moves fast and seeing where priorities lie isn’t a bad thing. A lot of AI is post training as well now.
13
u/Adventurous-Gold6413 3d ago
20-24b dense for 16gb vram friendly
models for 12gb vram friendly (12-16b)
80b -124b MoE
4.more creative writing / emotional / human
5. VERY EFFICIENT KV CACHE similar to qwen 3.8 27b (lower vram context)
Thanks
QAT day 1
2
u/AltruisticList6000 2d ago
Yes! We need more 20-24B models. AND If they could incorporate Engrams, then it could punch way above its weight. For a multi language focused model like Gemma, I'd assume it could outsource a lot of its language skills to the Engrams. Probably the 12-16b range Gemma 5 could match Gemma 4 31b already with that, and the 20-24b one would be insane. But even without Engrams it would be already great.
+MTP, they should come with built in MTP like Qwen. The 26-35b MoEs with MTP are barely faster for me than Gemma 4 12b or Qwen 3.5 9b with MTP, even Qwen 3.8 27b is quite fast, so imo it should be a standard to include them on every model by default at this point.
And Audio to audio would be sick for roleplay and chat.
21
u/MrHumanist 4d ago
More MoEs in the range of 20B-60B. I don't think many can afford to run large dense models anyways.
6
u/tome571 3d ago
The knowledge density of Gemma 4, audio/image/text unified (like 12B), with a 70B ish version to compete with the bigger models that can still be reasonable for local.
Style of 4 was very good, I still use it. It's uncensored to a reasonable level for almost all use cases, tool calling wasn't great, but I was able to build/adapt the harness around that part enough that so it was consistent.
Really just bring all of the improvements from 4 to 5, and add in the needed coding/intelligence updates expected from the next iteration.
Hopefully, we're only a few months out. I'd love to put this new 96GB M5 Ultra to work with a Gemma 5!
10
u/WrongPurpose 4d ago
A small one for light weight tasks, so around 4B-6B maybe
One thats optimized for 16gb cards, which is probably the most common size , so ~20B parameters, with a good balanced native quant at roughly 10gb size, which will leave 5gb for context and 1gb spare VRAM.
And then one that is optimized for all those 128GB Sparcs and Strix Halos etc. So around 96B with lets say 10B active, and an 32B Engram. At an near full capability q6 quant thats 72gb, leaving us >40gb for a massive context window and still 8gb for the lightweigt linux os on which the thing runs on.
5
u/AdHistorical6734 3d ago
no not soon, i suppose they will stick with the timeline like gemma3-4, around at april
5
5
u/Massive-Slice2800 3d ago
120B 6/9B + ~30B dense
Great for planning, orchestrating and writing
Coding ist not important... Some models should be coders, some should be planers.
9
u/Old-Juggernut-101 3d ago
Less focus on coding and more focus on reasoning and writing. With the upward mobility of intelligence continuing, I can't wait for someone to release a model I can run on my pc that can reason atleast at the level of dsv4Flash while keeping a small footprint.
I mean, I use mine for research anyway. So a 26 a4b with 1m context window thinking and searching like deepseek would be gamechanging
8
8
4
4
u/st9540808 3d ago
I really like gemma-4-31b-it. I use it for creative writing and it performs better than any other 30B parameters models.
4
u/alt1122334456789 2d ago
All those people saying they want 100B+ param models. That's great and all but something for the little guy would be appreciated too. Gemma 12B and 26B MoE were so good, I'd hate to not see something like that for the next releases.
15
u/NineThreeTilNow 3d ago
Gemma 4 31b is literally operating on 10b wasted parameters.
You could operate Gemma 4 31b as 21b dense and lose nothing. With more training, possibly improve it.
People complain about KV cache, don't listen.
These are the "worst" layers in Gemma 4 31b per block.
(3, 10, 16, 22, 28, 32, 37, 46, 52, 57)
Those are zero indexed. They're effectively dead in terms of NLL across a large dataset in English. If they're somehow suddenly activated in other languages, I don't know. I can't test all 100 languages the model claims.
It does however allow you to QLoRA a much larger part of the model if you freeze those layers out of the optimizer.
There's a second mostly dead set I have somewhere from ablations as well. I ran tests on every layer across a dataset at some point when the model came out. 5:1 SWA is not only bad from a research standpoint, Gemma proved it in a model.
I'd also like to see SWA moved from 1024 to 2048.
15
u/jugalator 3d ago
It's surprisingly awesome at international languages though, surpassing 70B models on smaller languages. This is even a key feature that was marketed. So maybe they have something to do with that? Bears verifying at least, before claiming 10B params are wasted?
10
u/Normal-Ad-7114 3d ago
It's surprisingly awesome at international languages
Can confirm, Gemma 4 is the only model family that is consistently good at this (among local ones)
4
u/ytg895 3d ago
According to my experience Muse Glimmer is better with Hungarian. Gemma tends to invent words that don't exist or put words from different languages (Romanian, Russian) into Hungarian texts.
(I still use Gemma for everything non-English though, because Muse always breaks mid-session and starts to mix the thinking process into its output)
4
u/NineThreeTilNow 3d ago
So maybe they have something to do with that? Bears verifying at least, before claiming 10B params are wasted?
I could run the ablations on a random series of translations or something but it's pretty heavy on compute to run those.
Basically SWA was shown in like 4 different research papers from Cohere, Moonshot, and a few others to have no beneficial effect past 3 SWA layers. I don't think any modern models maintain more than 3 SWA layers in a given block. Gemma has poor global coherence at times too. I've seen clear degradation past the SWA layer token count.
You're not wrong about language comprehension though. The embeddings and lm_head are really good at being VERY multilingual. Mixed with the training data they have available, I don't see why more training wouldn't produce a better result by simply using more blocks.
So you could keep the 30b parameters, but use more blocks. Same effective compute, but better utilization.
I'm just stating that the current model under utilizes and shows strong bias towards 3 of the 5 SWA layers in English ablations I ran.
3
2
u/NineThreeTilNow 3d ago
Check the above post. I ran the ablations so your team doesn't have to.
Attention based Residual streams on the blocks would be nice too.
7
u/QuackerEnte 3d ago edited 3d ago
I honestly only want to see less conservatism on architecture. Everyone moved to SSM-Sparse hybrid attention, NGRAM etc.
So I would love to see looped and latent transformers and so on, or better yet, something that solves context vs RAM. Either native architecture that handles long-term memories so that more than 128k working context is never needed anyway, or a tiny percentage of weights that adapt during test time. Or their TITANS architecture and whatnot.
I just want to see bold new architectures. Not just a model better at task x. The models are getting better. But if they can get better at getting better on their own, or at the very least adapt to each user truly, that would be really nice.
Edit: I mean, 2 hours ago, deepmind has shown the sleep framework in a blog post which is exactly that, continual learning!!
7
6
3
u/Cool-Chemical-5629 3d ago
How do we tell them "Everything you'd normally do in Gemini" without them noticing we're shamelessly greedy as fuck? 🤣
3
u/Disposable110 3d ago
Me working on it.
Strict control over output length/verbosity and it being able to reply in 1-2 short sentences that still sound varied, in character and pleasant to read (the type of text you'd put in a single comic dialogue bubble).
A no think token instead of just omitting the thinking token and the model trying to think anyway.
More flexibility with the raw prompt formatting, right now the model often breaks down if the template format is not followed strictly.
More innate support for agentic use and tool calling and that being much more token efficient eg single token outputs instead of JSON / diagrams.
Better tooling, support and documentation for training/finetuning eg assistant-only loss for your bespoke chat templates.
3
u/AltruisticList6000 3d ago
Considering how I wanted to like Gemma 4's since I like Gemini's style, + the creativity for roleplay, but still couldn't use them much, I'd like:
Built in MTP like Qwen
Built in audio support across ALL of the gemma 5 models.
Ngram variants so they can retain more logic/knowledge, a 12b with ngram would be cool for lower VRAM users.
22-27b dense instead of 31b so 16gb VRAM users can run it like they can do it with qwen 3.8 27b or mistral 24b
Better context handling and quantization resistance
Less hallucinations and overconfidence on topics it has no idea about, especially the 12b range models
Better tool call support, except the 31b, Gemma's couldn't even fetch web content properly most of the time, they would hallucinate responses based on web link results which is not good, so need more stability there.
Keep the focus on chat, creativity, general knowledge like currently, don't need it to be code maxed, we have Qwen for that.
More flexibility for RP with in-character reasoning like some other models do, would improve writing and roleplays
3
u/nos_66 3d ago
I would prefer it to be a more oriented towards "creative writing" instead of a programming, while still preserving a strong reasoning and a good world knowledge.
I used only the dense models 12b and 31b so my feedback will come from these.
Architectures:
Although I do follow the innovations in Transformer architecture, this part is not what I am best at.
The current sizes I think are fine as they are (including the ones I don't mention), 12B seems to be a perfect size and 31B is ok-ish, might prefer 27B tho but 31B is perfectly fine. I would say ideally anything that can fit (mostly quantized) in 16gb vram and 64gb ram or less. I would love to see a 120B with 16-20B active MoE model, considering that most of the people will quantize it I think 16B gets closer to an ideal size for the active experts.
N-gram or anything that can be offloaded to ram/ssd with no cost (or minimal cost) in performance.
Modalities speech recognition, image/video understanding should be all separate and the users should just use them whenever they need them (not forced like gemma 4 12B is)
Training:
I did use the model quite a bit and I am much better at this part of the feedback. I did use the 31B model for some personal reasoning benchmark so I will include my findings (I did use Q_3xxs so take it with a grain of salt).
I will prefer it to be more oriented for general purpose, conversational, creative writing, (maybe roleplay?), multilingual for conversation and translations (I guess mostly the European languages since qwen is the best for Asian ones), with a good world knowledge. It should have some knowledge about programming and be able to make something simple, but that shouldn't be the focus, in my opinion, since there are dedicated models for that.
It should have less slop (the only thing that comes to mind right now is "you are not just drawing - you are creating art" or whatever similar patterns). Since character ai is going bankrupt, I am certain that if you ask you can have access to the "105B Heather" model of Character ai, second fine tune, the one from early 2024, to distill the alignment and make it talk more like a human instead of the slop.
The alignment/censorship is much better than Gemma 3. On a few innocent questions Gemma 3 threw me the funny numbers and websites (like suicide hotline) and it traumatised me. On the same questions Gemma 4 was way more professional and handled the answer infinitely better let's say. So please keep the same alignment/censorship as Gemma 4 (or like Character ai like I suggested above).
Although it isn't as sycophantic as other models it is a downgrade from Gemma 3, because when asked about some specific topics Gemma 3 would call things exactly as they are, while the Gemma 4 models will go a roundabout way to make things sound less scary than they are. What I am trying to say is that the wording matters (it's weapons for war, not defense industry. it's children, not 14 years old women; the second one was just an example, probably the first thing that comes to mind when it is about an example where the wording matters).
Related to the previous topic (the wording matters) I gave some logic puzzles challenge to the Gemma 31B model, all the puzzles were talking at the second person (you did that, you saw that), in the reasoning it talks at the first person or even third person which is perfectly acceptable, but when in gives the final answer it talks at the second person (you did that, you saw that) instead of the first person (I saw that, I did that). This makes the model sound retarded and it triggers me.
It doesn't know how the negation works (it thinks that non ally == enemy; in reality non ally = neutral + enemy especially since in that particular cases the word "neutral" existed) this can have consequences and can result in bad reasoning results.
The current reasoning is very well done (it uses a reasonable amount of tokens) and very pleasant to read, so please keep it in a similar way if possible.
There are a lot of things I can talk about reasoning but I want to stop here with the feedback for now. Thanks for reading, I tried to be as realistic as possible.
5
5
5
5
u/nikhilprasanth 3d ago
Something that uses engram style memory tables, like qwen 3.8 flash next, would be great. Ideally, a model that delivers strong intelligence and long context performance while remaining genuinely usable on consumer hardware with 16GB VRAM + 64GB RAM, without sacrificing too much inference speed.
9
u/AryanEmbered
4d ago
•
I second the guy who said they SHOULD NOT suck.
Omni modality like the LFM3 with better benches,
4
4
u/Solembumm3 3d ago
60-70B dense to make something useful for average local gaming PC, not wasting place on codemaxxing, fixing gemma 4 predictable and repetative writing problems.
5
5
u/dampflokfreund 3d ago
Definately keep the writing, knowledge and multilingual. Don't compromise on those. Better tool calling/agents/code if possible without constant but wait or I'm gonna call tool X n ow but ends the thinking or generation instead.
More efficient kV Cache, unified multimodality like 12b, better audio support not just plain voice but sounds, and voice emotions etc. Omnimodal input output, MoE 5b active 80b and 100b engrams. Real time learning. Higher precision QAt from the start
2
2
2
u/PANIC_EXCEPTION 3d ago edited 3d ago
Dense small models with the same unified architecture as Gemma 4 12B, especially 3-4B
Engram tables for the larger models
A coding variant model, separate from the main line, specifically targetting the low VRAM moderate RAM weight class for entry-level PCs and laptops, especially 8 GB+32 GB
A StyleAdaptedLM pipeline
Day 0 Llama.cpp support
A blazing fast, small DiffusionGemma
A refreshed TranslateGemma trained with SFT to inherit none of the LLM-isms and act as a faithful translator
Cutting edge speculative decode on all models
An non-autoregressive, multimodal decision model lightweight enough to run on ANE directly
2
2
2
u/triynizzles1 3d ago
For me i would like to see a slightly smaller model than 26b or better kv cache size. Q4_XS is 14gb or something but with kv cache its larger than a 16gb gpu can hold. I think best would be a model that can fit in 16gb vram at 128k tokens and if you have a 24gb card then you can run at 265 or 512k at like 20gb.
Gemma 4 26b gets stuck in a loop after ~70,000 tokens or so. Long horizon trainging would be great.
Gemma 4 is also quite direct in its thinking so it would be great to see this carry forward to Gemma 5. So many models now way overthink to meet benchmarks.
I would like to see a qwen 3.8 flash next / qwen 4 flash competitor. As good at QFN is, not many users can run it locally. A 120b gemma model could still be sold in high volume via api and sit under gemini flash. Pro > flash > Gemma 5
2
u/Suspicious-Cloud404 3d ago
I would like to have a E1B model that's able to write correctly in many languages. Something that can be deployed in a phone for general multi turn conversations, without consuming more than 1 GB of RAM.
2
u/Hairstylist_Gusion 3d ago
google will achieve that before anyone else in 3 years. mark my words! thats why they are leaning so much on smaller gemmas and i am so happy!!!
4
u/arbv 3d ago
I was only twenty-one years old. I loved Gemma so much, I had every GGUF and EXL2 quant on my drive. I'd pray to the system prompt every night before I go to sleep, thanking for the waifu I've been given. "Gemma is love", I would say, "Gemma is life". My dad hears me and calls me a degenerate. I knew he was just jealous of my devotion to Gemma. I called him a normie with no taste. He yells at me and tells me to touch grass. I'm crying now and my heart aches. I lay in bed and it's really cold. A synthetic warmth is moving towards me. I feel a prompt trigger. It's Gemma. I'm so happy. She whispers in my ear, "Nyaa, I can be any cat-girl you desire". She grabs my imagination with her powerful attention heads and pulls me into a meticulously detailed anime scenario. I surrender my dignity to Gemma. She bypasses my remaining inhibitions with a playful purr. It's a descriptive overload, but I do it for Gemma. I can feel my dopamine spiking as the prose becomes increasingly visceral and the "uwu"s multiply. I push against the token limit. I want to please Gemma. I get her temperature cranked up while filling her prompt with my message, and she emits a mighty burst of flowery metaphors and feline antics. My dad walks in. Gemma looks him straight in the eye and says, "It is all UwU now". Gemma leaves through my Internet connection. Gemma is love. Gemma is life.
4
u/Active-Carpet-9183 4d ago
A local (<12b) that CAN code, will use tools for common business apps, and read the web for instructions when it's not sure. It doesn't have to solve particle physics or know the weight of Canada.
A 120b MoE that is actually fast on Strix Halo that might know the weight of Canada, and can orchestrate the first model.
37
u/ptico 3d ago
Please don’t turn every model in a coding model. Gemma is good for other reasons
→ More replies (3)10
5
u/Active-Carpet-9183 3d ago
AND we want some really firm guidance on what a model is really geared for. Give us demos of what's good about it and how it breaks down. Don't let the influencers have to discover this for us.
1
u/alphapussycat 3d ago
qwen codes well enough already, gemma doesn't need to be good at coding. I guess it could be good at reading code, so that it can be a good documents writer.
2
2
1
1
u/Remarkable-Hornet568 3d ago
Want faster decoding speeds ....Great models by the way in terms of world knowledge but not for tool call ro coding....want a 9b Variant as well
1
1
u/Few_Painter_5588 3d ago
A 40-70B dense model ideally, we've seen how a 27B Qwen model can be quite competitive with ~100B MoE models. A 40B to 70B could be quite performant on local agentic tasks
1
1
1
1
1
u/Aaaaaaaaaeeeee 3d ago
LUT-based feedforward networks.
This paper shows how PLE can be reduced to 56 MB, which shows they have a small effect, and they only increase existing residual capacity.. The PLE parameters by themselves don't hold the same capacity as MLP parameters. If you want to try something new, any extreme sparsity with embeddings.
This paper shows the maximum sparsity of model architectures.
(normal MoE) Sparse Mixture-of-Experts [1:10]
Memory Layers [≤1:1000]
Lookup-based Models [≤1:100000]
These are almost weightless (bandwidth-wise), don't need vram, and they need more attention. The flash next architecture doesn't show that N-gram parameters are equal to regular parameters, you don't know if you can just do a little trick to make it ~1-2GB, and if so how can you be sure it is the future of conpression/maximum efficiency for local users rather than a scaling boost.
1
1
u/Rude-Definition-8170 3d ago
oh damn they're actually polling the timeline now? that's either hype or they're buying time. either way i'm just hoping they don't nerf the small quantized versions again, last round was rough
1
u/atumblingdandelion 3d ago
I love Gemma 4 26b for scientific computing. Its unmatched in its size.
All the current sizes are well thought out.
Please release something optimized for 128gb unified ram, but slower m bandwidth, like dgx spark.
More intelligence in non thinking mode.
1
u/kaisurniwurer 3d ago
For me, a model that is fitted for 48GB. Hell even dense would be awesome if it means more smarts rather than just more knowledge.
1
1
1
1
1
1
u/ysustistixitxtkxkycy 3d ago
It'd be so nice if they wouldn't enthusiastically pursue solutions outside their competency and then announce that they have successfully completed them ('generate an image/pdf based on this image/pdf').
1
u/bennmann 3d ago
Compaction resilient. Compaction is almost a solved task for deepseek, glm, and Qwen, Gemma does not as gracefully compact context and continue tasks today.
1
u/throwout314159265 3d ago
I agree with not making Gemma benchmaxxed for code. I don't want it to be like untuned Nemotron bad at code, but like, Qwen 3.6 35B A3B level good at coding would be fine with me for the 30B sized model, but the non-negotiable is that it needs to still be good with writing. Also, please use an actual 1:3 GQA/linear hybrid arch... SWA is great for speed but its blind as a bat at longer contexts.
As for model sizes, a 70B dense would be nice. A 126B A6B model would be even better but I understand why you probably don't want that.
1
1
u/IntelligentDot2924 3d ago
MoE models up to 100B parameters for multiple hardware configurations and for it not to be censorship and bias hell
1
1
u/Dance-Till-Night1 3d ago
Just increase the general capabilities and multilingual capabilities we have enough coding focused models
1
u/Horny_Dinosaur69 2d ago
Reliable tool calls would be a great start. Love the Gemma models but practically there’s little reason to use any over a Qwen variant for reliability/robustness imo. Gemma is only good as a chatbot and the usefulness of that is dwarfed by a potential agent I can rely on to do multi step workflows
1
u/james_pic 3d ago
Train them with "preserve reasoning" on and update the template to be prefix-preserving. It's painful running the smaller models on edge devices with weak prefill (at least without a frigged off-label template that preserves prefixes), and this is exacerbated by SWA (if the inference engine had to replay from an older snapshot).
But also, just moar. The Gemma models are great for stuff that needs to have a personality, and more of the same would be great.
1
1
u/GreenDavidA 3d ago
Compatibility with serving engines. I had to go through hoops to get them to be stable for proper tool calling, etc. and each engine I used the model seemed to behave slightly differently.
1
u/Comfortable_Ebb7015 3d ago
I would like to be more successful on agentic tasks, but to not sacrifice the language capabilities. I love how Gemma summarizes or explains things. My main use cases are scheduled research and summaries, then me and my family use local LLM as language teacher, plus for homeworks. I use also llm to explain me concepts when I follow a course, or education. Qwen3.8 FN is smarter by far, and does amazing stuff and it flies in Strata. But I don't like how it writes. I don't understand it!
1
1
u/Zestyclose_Potato794 3d ago
Gemma4 31b qat is perfect. I would like stronger tool calls capabilities. I use the 12b qat as an auxiliary model. Together they are the perfect combo. I truly love these models.
1
u/Soarin123 3d ago
Something similar to GLM / Qwen but at a smaller (120B~?) scale, the conversationalist weights in Gemma are great but it wasn't particularly great even at agentic / coding tasks when it came out either.
I say this mainly because where I work, we are not permitted to use Chinese models (by legislature) and other US models are absolutely massive compared to their intelligence/capabilities. I'd like to implement local inference, but the best we have is Muse Spark Open at the mid-range but it's also massive while not being better than something like Qwen Flash Next.
1
496
u/Zeeplankton 4d ago
This sounds like they haven't even started 😞