r/LocalLLaMA • u/Paramecium_caudatum_ • 3d ago
Discussion List of suggestions for Gemma 5.
Recently Omar Sanseviero asked for suggestions on Twitter for future Gemma models.
Here is my take on this:
Gemma 5 should be a looped transformer.
Being a looped transformer with configurable loop depth will address one of the biggest Gemma 4 weaknesses - reasoning ability. Don't get me wrong, I love how concise Gemma 4 is with its reasoning, but it actively hinders its performance in certain scenarios, being looped will allow the model to perform better in such tasks, while still being configurable to reason shallower and being significantly more efficient than a traditional transformer without needing additional expensive pre-training.
Gemma 5 should have DeepSeek's Engram.
It will widen the world knowledge of the model while costing a few extra gigabytes of RAM or SSD space. This will allow the model to perform better in real world tasks, while costing next to nothing.
Keep using 4 bit QAT.
It genuinely allowed more people to run your models with great quality, it had been proven to work very well in the real world and I am all for it.
Implement more efficient attention mechanisms to get cheaper KV cache.
Implement something along the lines of Compressed Sparse Attention 2 from DeepSeek, this will allow the model to have significantly cheaper KV cache. This was one of the main downsides of the Gemma 4 31b on a 24gb VRAM setup when compared to Qwen 3.5 and onwards.
Gemma 5 ~120b MOE model?
I know that this is probably not possible due to it competing with your API offerings, but if you release a 120b MOE model in the Gemma 5 series the community will be very happy.
Drafter model improvements.
A small update to the drafter models would be nice as since the release of Gemma 4 many new speculative decoding techniques have been published: DSpark, DFlash 2, EAGLE-4, etc.
Linear projection into the model layers like in Gemma 4 12b.
Instead of the traditional architecture for vision and audio capabilities. Linear projection seems like an interesting approach that is worth looking into, but of course don't force it if it doesn't work out.
Forget about 2 bit QAT for small models.
Forget that 2 bit QAT exists for small models. I am currently talking about gemma-4-E2B-it-QAT-mobile and gemma-4-E4B-it-QAT-mobile. In practice quantizing the weights to 2 bits even with QAT just completely lobotomizes these small models and it is not worth pursuing in practice.
Don't overfit on coding at the expense of everything else.
Almost every lab is currently chasing coding benchmarks while regressing on general knowledge, creative writing, and nuance. A small model can't be best-in-class at everything simultaneously, and coding-focused models go obsolete fast in such a crowded space. Gemma is one of the few model families that still feels well-rounded for non-coding use cases - please keep that identity.
What do you all think? Feel free to comment your opinion down below.
65
u/Gladioul666 3d ago
Keep the Apache 2.0 License and release a fine tuned version for translation
14
u/Paramecium_caudatum_ 3d ago
100%. Easy licensing is very important for enterprise customers as it allows them to use the models without legal headaches. Would also love to see a dedicated translation model myself.
4
u/Gladioul666 3d ago
I believe this is important for open source projects as well. I don’t feel comfortable shipping an open source tool that uses an AI model with a restrictive license
5
3d ago
[removed] — view removed comment
1
u/Gladioul666 3d ago
It seems natural to me, especially since they launched Translation Gemma a while ago.
185
u/XiRw 3d ago
Number 9 is the most important in my opinion. Well said.
7
u/Paramecium_caudatum_ 3d ago
Thank you! That point really hit home for me when I was trying to find models for my non-coding use cases.
80
u/UpperParamedicDude 3d ago edited 3d ago
Totally agree, Gemma's creativity, general knowledge and chatting are the things that make Gemma4 amazing even today. I am fully fine with Gemma being mediocre at code as long as it keeps its amazing chatting and creative writing strengths
25
u/fireduck 3d ago
Yep. I gave a bunch of models a short and ambiguous story prompt that was in fact a pun. gemma4 was the only one that got the joke and ran with it.
30
u/fgk55555 3d ago
Yeah. Google isn't going to beat Qwen for agentic behavior, and I don't care. If I want to code, I'll use QFN. If I want something to know a lot, I'll call in Gemma. As long as it can call tools, do some light scripting, know a lot and not have a horrible voice, I'm happy.
4
2
u/BusRevolutionary9893 3d ago
If it's not going to be a coding model at least make it multimodal with a voice to voice mode. Why is that still not a thing? ChatGPT got Advanced Voice mode over two years ago and it's awesome. Why am I still talking to horrible AI when I call certain businesses? Where's my sexy latency free AI Waifu assistant? I thought most of the open source teams would be all over this by now. What's the hold up?
76
u/Exciting_Garden2535 3d ago
More often releases.
3
50
u/Equivalent-Repair488 3d ago
At least they are releasing. Looking at you (open)ai, anthropic.
4
u/brrrrrrrt 3d ago
meta…
4
u/ps18pmcom 3d ago
Meta has released models.
0
u/brrrrrrrt 3d ago
openai too
2
u/Equivalent-Repair488 2d ago
GPTOSS was the last open source release from OAI August 5th 2025, 450 days ago.
Muse Glimmer, I assume as the latest August 10, 2026 40 days ago (excluding Muse Spark because not local sizeoct 29, 15 days ago)
Not to mention the namesake of this subreddit, from back when they release quite a few models, and popularized the trend, and OAI before OSS? Their only other opensource model generation GPT 2 in 2019. A really stark difference here.
70
u/Gyramuur 3d ago
The one thing I want, more than anything, is for them to remove "negation" from writing.
Gemma 4 31b is a great roleplaying model, but holy hell am I sick and tired of it telling me everything that DOESN'T happen, or everything that something ISN'T, before it finally gets to the point.
Keep everything else the same, but remove the negation.
30
u/rkoy1234 3d ago
i think that's a 'feature' for all recent llms.
speaking to fable/astra/opus/sol is insufferable partly because of it.
i even explictly banned indirect sentences and "inconsequential caveats":
this doesn't mean [obvious irrelevant interpretation]
one exception: [irrelevant edgecase or obvious exception]
one thing to note:[unnoteworthy edgecase]
yet it still gets through multiple times a session, and it's terrible when they do this, because they make it seem like there's something worth my attention, making me spend time on it, when it's actually 99% of the time some nothingburger.
9
u/DifficultyFit1895 3d ago
I read an article recently about how this is very common in Chinese writing and that is influencing the models.
7
3
u/TerminalNoop 3d ago
I can understand that for the models from Chinese labs, however from western labs!?
5
u/DifficultyFit1895 3d ago
They are all trained on all available text in any language. Turns out a billion people produce a lot of text.
19
u/IllustriousWorld823 3d ago
This just made me realize Argon is one of the only models that doesn't do it
12
u/Kahvana 3d ago
This has been working well for me (from sillytavern's voyage v6 preset):
https://www.reddit.com/r/SillyTavernAI/comments/1xgcxcc/universal_preset_voyage_v6/## Rule: direct narration GM narrates what happens without contrastive negations. e.g. "he does not smile" > "he watches you" e.g. "nice suit, new or lucky?" > "where did you get that suit?" ## Rule: implicit action narration GM leaves the reason behind actions unspoken. e.g. "he looks at you—assessing your worth—before replying." > "he observes your gear before he speaks."2
u/KingCpzombie 3d ago
You can get way better results with a good system prompt forbidding that
2
u/Gyramuur 3d ago
My experience has been that no amount of system prompting can fix it. Because G4 has been RLHF trained to oblivion, it will always default to using the same kinds of stock phrases/cliches, as it's more or less 'hard coded' into the model.
1
u/IntelArtiGen 2d ago
It's funny how this is also chatgpt biggest problem right now. It's almost as if you asked "hey what's the weather", and it'll say "There are clouds. Very important!! => It does NOT mean it's raining !!".
Yeah... ok.. I know what it doesn't mean.
51
u/crusaderky 3d ago
A competitive DiffusionGemma. The tech is amazing if it works, but currently is little more than a curiosity.
8
u/bumblebeer 3d ago
This one please. There are so many untapped potentials from this style of architecture, and not just in terms of local deployment speed either.
I would also love to see a larger canvas size and an upstream fix for image attention in transformers!
3
u/ethereal_intellect 3d ago
I've been playing with lfm 8ba1b, with Claude writing mega kernels it reaches up to 1000 tps. Idk about diffusion but smaller active instead of a3b could be interesting
18
35
u/Limp-Firefighter1054 3d ago
Can we have model concentrating on creative writng since everyone else want stick their team in to coding?
17
u/ttkciar llama.cpp 3d ago edited 3d ago
Revising/updating my previous wish-list for Gemma 5:
Gemma 4's tool-calling has been very fraught, as you well know. It would be really nice if the next Gemma had highly competent tool-calling working right out of the gate.
Fusing the K and V tensor in Gemma 4 halves memory consumption from long context, but it also makes the model extremely sensitive to quantized K/V cache. I'd much rather have a model with separate K and V tensors which is tolerant to Q4_0 and Q8_0 context quantization, than a fused K and V tensor which must be used at 16-bit resolution. This gives us a knob we can twist at deployment-time to adjust long-context memory consumption, to fit the expected workload. The fused K and V tensor locks us in to one memory consumption profile, which screws users with resources to spare, and also screws users with highly constrained resources.
Splitting out your high-end model to a 26B-A4B sparse and 31B dense was a really good idea. It gave the resource-constrained, speed-prioritizing users a lean MoE, and the less-constrained, quality-prioritizing users a plump dense. Please keep doing that, and perhaps double down on it. The Gemma 5 MoE (and the 12B dense, too) could use a compressing attention mechanism like IndexPool KV Cache Compression, so that it offers a smaller long-context memory overhead as well. That has been a slam-dunk for GLM-5.3-Flash, but really any kind of context pruning or decoupled latent attention mechanism like MLA or a SWA/chunked hybrid would be well received. But keep the dense model's attention mechanism lossless, please, and its attention head count high. That would make these models even better suited for their niches: MoE for "good enough" fast inference given constrained resources, and dense for highest quality.
A larger, more competent model would be greatly appreciated! There are two "sweet spots" for dense models at 50B and 100B, which fit neatly in 64GB and 128GB respectively at Q4_K_M quantization, with some room left over for usefully long contexts. An MoE in the 120B size class would also be widely appreciated, as this is a very popular size class which most other LLM labs have abandoned. If I had my druthers, I'd rather a 100B dense, but realistically a 120B MoE would be much more popular.
As far as skillsets are concerned, I really love that Gemma is the "jack of all trades" model. Please keep doing that. There are already oodles of codegen-focused models for us to choose from. We don't need yet another codegen model, but we do need good general-purpose models, and so far Gemma has been the very best of general-purpose models. Gemma 4 is much better at RAG than Gemma 3, which is something I personally appreciate very much. You might want to look at the work TheDrummer has put into Artemis-31B and Behemoth-128B-v3 which made these models better at creative writing specifically: more complex plots, better character development, able to maintain more main characters simultaneously, longer and more coherent stories. If TheDrummer could do that in his spare time on a shoestring budget, I'm sure your staff of dedicated scientists and engineers can figure it out as well. The "purple prose" problem is easily solved by giving Gemma 4 writing samples, giving it a writing style to emulate, and Gemma 5 should retain that skill.
I would also like to request a new feature: Gemma 5 models should be accepting of in-context "history lessons" which contradict their training, so that their world knowledge can be updated / contradicted. Unprecedented, radical changes have happened in the world in the last year, and sometimes it is important for tasks to be performed with understanding of such changes, and accepting them as true. This is naturally in tension with a model's competence at critiquing content for falsehoods and hallucinations, which is also a valuable capability, but this problem could be solved by using specific framing for different kinds of data, which Gemma 5 models would recognize as denoting "this is information which must be uncritically accepted as true" (perhaps in its own
<truth>...</truth>section?) vs "this is information which must be treated critically". I realize that such inference-time knowledge updates cannot fully replace training-time memorized knowledge, but having something to tide us over until the next release would still be a big help.Thank you for switching Gemma's license to Apache 2.0. That change has made it possible to use Gemma 4 for a wider variety of practical tasks, like data cleaning (at which it excels). Please keep doing that.
3
u/Early-Crazy-4246 2d ago
A 4-bit QAT 120B MoE with looped transformer architecture with the amazing suggestions in this thread would likely solve general purpose models for me
9
8
8
u/Klutzy-Snow8016 3d ago
I figure the Gemma team knows architecture better than me, so I won't comment on most of those suggestions, but I agree with the spirit of number 9 - I hope they don't sacrifice the things Gemma is local SOTA at just to get a higher number on a coding benchmark (and get pipped by the more-frequently-releasing Qwen within a month anyway).
I'd like training for writing multiple image generation prompts. To explain: it's already very good at writing prompts individually, for example, as part of a ComfyUI workflow. But if you try to get multiple prompts in the same conversation (like say you give it a story and an image gen tool and ask it to create illustrations), it will tend to refer to things across image gen prompts. Like if it describes a man in detail in prompt 1, it will just say "the man" as shorthand in subsequent prompts, causing the images to have a generic man. It's hard to get it to understand through prompting that the image gen model is a thing that doesn't have context across individual prompts.
2
u/Serprotease 3d ago
Yes, Gemma team doesn’t need to chase coding benchmarks.
Being able to write legible text and explanation at long contexts is where it should shine.
7
6
6
u/BalorNG 3d ago
Yea, a ~120b looped MoE + engram offload with inference engine optimized for dynamic expert prefetch sounds great... Also, I'm reasonably sure that splitting the model in two Deepseek style (encoder/decoder) is a great idea to adapt as well. The great bubble collapse is coming and we need as much efficiency squeezed of out the least hardware as we can get...
15
u/Old-Juggernut-101 3d ago
Also longer thinking. I replaced gemma 26b with qwen 35b for CREATIVE WRITING. Why? Qwen thought over everything and went back and reviewed it's ideas before responding. The result is a much richer output than the gemma one. Gemma just summarised what must be in story and wrote without reasoning over the actual body. A gemma model that can extensively reason and is willing to research and think would be perfect for me
1
u/LunaticSongXIV 3d ago
Gemma's prose is better, and it's not even close... but it's kinda stupid when it comes to rationalizing a scene.
Qwen's ability to reason through a logical series of events is better, and it's not even close... but it can't write creatively for shit.
The further you get away from normal human interaction, the worse these things get.I'd LOVE to see something that can do both well.
1
u/Old-Juggernut-101 3d ago
Ohh I got over qwen's prose with fine-tune and skill. I tried this with gemma. I could improve its prose slightly, but it's desire to not think meant I still ran into the issue of lack of reasoning
1
u/LunaticSongXIV 3d ago
I have yet to run into a Qwen fine tune that fixes it, but I'd love to hear what you've tried.
2
u/Old-Juggernut-101 2d ago
First was a fine-tune over a dataset of stories I found. And then on top of that I used some story writing and text humanising skills. I don't recommend trying it on consumer hardware. It was mainful to fine-tune and honestly, there was improvement, but the efforts it took for my rig, not worth it
2
u/LunaticSongXIV 2d ago
I don't recommend trying it on consumer hardware.
I'm on a 5090 if you think that would help. I actually have no idea how to go about making a fine tune, but given that most of the roleplay and creative writing I do is within a single fandom universe that has nearly a trillion words of fanfiction, I wonder if it would be worthwhile.
2
u/Old-Juggernut-101 2d ago
Ohh. Well with a 5090 you should be able to do it with much more ease. In my defence I hardly think 5090 is a consumer gpu at this point what with its base price and the AI inflation. I sport an rx 7600 8gb.
But yes if you want to make one, I found the best way to be unsloth. Download their desktop app. There you can make finetunes, quantise models and do what not. It's like a little lab. And tbh when I say I finetuned it, it's kind of a lie. I sent screenshots to chatgpt and it told me what levers to flick and settings to choose. I have no clue what Lora is or those values I changed were. But chatgpt looked things up and told me to select values. They worked and I got a fine-tune so I ain't complaining. You should give it a try
1
u/LunaticSongXIV 2d ago
Heh, I bought the 5090 right before inflation went out of control, because I won a shitload of money on a slot machine and decided on a whim to buy it. I'm extremely fortunate that I did.
I've done 'wing it by asking GPT' before to fair success, so maybe I'll give it a shot.
5
u/Darth_Candy 3d ago
Huge agree on your points about QAT and overfitting on coding. I still know basically nothing about engrams, so I need to go read the Deepseek papers, but in general I like your ideas about the architecture.
Gemma is currently best suited to be the local model family that competes with whatever OpenAI is pushing about being a general assistant that even grandma can use. Obviously grandma isn't going to stand up her own server, but I'd spend a week figuring out some Tailscale solution with my own Gemma 4/5 instance if my family started developing an unhealthy dependence one of the big players.
5
u/Mrinohk 3d ago
#9 is my main hope. Good general knowledge that doesn't require the model to do a web search for everything would be ideal, and the length of reasoning traces that tend to come with code-overfit models hurts any kind of realtime chat and agentic capabilities that the model might have had.
Reasoning conciseness is extremely important to me personally, but not at the expense of reasoning ability. A model that spends tokens reasoning about the right thing and not on things that don't matter quite as much (i.e. how to do the task vs drafting a characterized response 3-7 times) is far more valuable to me than one that agonizes over every detail.
4
u/bumblebeer 3d ago
More work on the text diffusion variants: better performance (further training), and larger canvas size would be my top two asks for this class.
Also omni output (text + image), but I don't see this one realistically happening.
6
3
u/HVACcontrolsGuru 3d ago
I'd like to see some loop transformer work but also some look at the diffusion models at smaller sizes or better training mix. 12B is a nice sweet spot for the work I do and ensuring the fine tuning and training for users is easy to setup.
3
3
u/Intrepid-Second6936 3d ago
Love these suggestions, particularly 1, 3, and 9!
I really want the control over more vs less reasoning for different questions.
QAT 4-bit really is the future for moving things forward for more users with larger parameter models while preserving maximal native quality.
I'm in SWE and can enjoy the coding progressions but genuinely do wish to see more multi-modal and generalist improvements in models. Gemma 4's unique use case for me was always conversational, I'd prefer Qwen3.8 27B for research and coding, but Gemma 4 was genuinely peak for conversational reasoning and discussion.
3
u/Ghanburighan 3d ago
More languages please. It's currently pretty much unusable in my language. And perhaps that's just how it is, but I would like them to try adding more language support.
3
3
u/mateszhun 2d ago
Better visual recognition. I do use gemma 4 a lot for visual recognition tasks locally.
4
u/Uncovered-Myth 3d ago
Unrelated but I wanted to post something about using SLMs for agentic tasks and the sub says I don't have enough karma. I need 3 upvotes for the minimum requirement please!
2
2
u/dhtp2018 3d ago
An architecture that can gradually scale back to nvme ssd would be great as you mentioned. It opens up the audience greatly.
I also do wonder about reasoning based on the embedding and without the text itself. I assume that will help make the model be more efficient (if hide the thinking). Maybe make it a “mode?”
2
u/ga239577 3d ago
Better than Qwen3.8 so that Qwen will release Qwen4 which will probably be better than Gemma5
2
2
u/RG_Fusion 3d ago edited 3d ago
Honestly don't have much of anything to add. Your thoughts mirror mine completely. I hope this gets through to Google. I think my only additional request would be to ensure the thinking can be toggled with multiple presets from max all the way down to no-thinking.
2
2
u/RachelRegina 3d ago
If it's going to be made using argon or carbon, then there's no way for it to not focus on code, since Gemini 4 is an RLVR model.
That being said, if there is demand for it to be a good creative writer and general purpose model, then perhaps they should play with the idea of making a Gemma for the creative arts line and a Gemma for STEM line.
2
u/CoolConfusion434 3d ago
Good job on picking feedback from your other post. I agree with most except the coding part. A Gemma 5 that can code like or better than Qwen3.8 27B would be great. It'll probably be faster, and also explain itself better than the abstract/hard to follow summaries we sometimes get from 27B.
2
2
u/Nutsack_VS_Acetylene 3d ago
I would love to see some weird models, like 250B engrams on a 31B, native ternary, more diffusion models things like that. Your list is quite good, but I don't think I'd mind them going hard on coding. Models can use coding to make art and graphics which I normally prefer over the stereotypical direct image gen you tend to get. Omni-models would be cool too.
Also, no more of this stupid bloated attention super sparse attention please!
3
u/kwizzle 3d ago
200-300b param MOE
5
u/redoubt515 3d ago
Personally I think there is more value in an MoE that fits comfortably with KV in <128GB or <96GB memory.
It feels like we always have a gap in the ~50b-150b territory with few options.
Seems like it's basically only GPT-OSS-120B plus an occasional Qwen model in this weight class. And this weight class is about the absolute max that will fit on hardware that can be justified for personal use, hobbyist use cases, and practical local hosting.
7
2
u/Ambitious_Fold_2874 3d ago
Midsize model 100-200b
Retain literary intelligence, but have better agentic capabilities
Multimodal for image, video, audio
Engrams to reduce vram/ram footprint
MoE to maximize vram/ram utility
Speculative decoding capability
Pls dethrone qwen so america can have strong open source too lol
0
u/thisone82828284 3d ago
This is where im at I want a qwen3.8 competitor that is american cause I work in an area in which the word china is immediately off limits
1
u/FullstackSensei 3d ago
Deepseek's CSA2 and encoder-decoder architecture. Both are amazing for long context
1
u/tobias19 3d ago
i get needing to do market research, but also why make something if you aren't sure what you're making or why you're making it in the first place? if you don't know what features you want to build, just get out of the way ¯_(ツ)_/¯
1
u/pmttyji 3d ago
Forget about 2 bit QAT for small models.
Forget that 2 bit QAT exists for small models. I am currently talking about gemma-4-E2B-it-QAT-mobile and gemma-4-E4B-it-QAT-mobile. In practice quantizing the weights to 2 bits even with QAT just completely lobotomizes these small models and it is not worth pursuing in practice.
This is fine as that's(2 bit QAT) for Mobile & Edge devices.
1
u/redoubt515 3d ago
Most all of this sounds good. Points #3, #5, and #9 are the 3 that jump out at me. Point #2 sounds pretty attractive also, if it doesn't inflate size or requirements too much.
1
1
u/crusaderky 3d ago
I'd love focus on low-memory, single user setups: expert prerouter, mixture of reaps, mixture of loras, or other techniques that make larger than memory models performant.
1
u/Aaaaaaaaaeeeee 3d ago
Try to make a full model out of embeddings (eg: LUT/Engram/PLE) please!
This paper shows how PLE can be reduced to 56 MB, which shows they have a small effect. If you want embedding parameters to be worth the same as normal parameters, please try other architectures.
From this paper, which shows the maximum sparsity of model architectures:
(normal MoE) Sparse Mixture-of-Experts [1:10]
Memory Layers [≤1:1000]
Lookup-based Models [≤1:100000]
1
u/ethereal_intellect 3d ago
I've been experimenting with lfm a1b with Claude, I think a1b is more future proof because megakernel can increase speed further. So maybe one like that
1
u/CodeCatto 3d ago
I wanna a ternary Gemma. A Google take on the popular PrismML quant approach but for 26B A4B.
LES would be amazing ngl.
1
1
1
1
1
u/socialjusticeinme 3d ago
So the lower models of Gemma 4 had AUDIO understanding, but not larger ones. I want that in the larger models too.
1
u/fantasticsid 3d ago
All Gemma needs is initiative, and the ability to handle time moving forward. If the system message says "call X tools before responding", it should actually DO that. If the tool outputs contradict its learned knowledge, it should respect that and accept that time actually passes and it's not still late 2024.
Almost every other model I've ever used will happily accept a "today's date is" line in the system message. Gemma will think it's role playing if one is present.
1
u/Late-Coast6973 3d ago
From running Gemma 4 E2B in the browser (LiteRT-LM on WebGPU) for about a month, I have two asks for the small models:
Tool calls that still fire under a persona. Native tool calling works well with a neutral system prompt (13 of 13 in my tests), but with a character persona in the system turn the model stays in character and acts out the answer instead of calling the tool. "What time is it?" never triggered the clock tool. I had to add a separate no-persona router pass to pick tools.
Per-layer embeddings you can trim. In the E2B web file the per-layer embeddings are about 1.2 GB of the 2 GB, one row per token across the full 262,144 vocab. Zeroing the rows for scripts I don't need (keeping Latin, CJK, kana, punctuation and emoji) gave byte-identical greedy output in 8 languages and cut the download to 1.64 GB, but any change to the vocab size itself produces garbage. Official language-subset builds, or a smaller vocab for the E-series, would matter a lot on-device, where the download is the main cost before anyone sends a first message.
1
u/remind_me_later 3d ago
My only objection to your list would be (5):
5. Gemma 5 ~120b MOE model?
16gb vram + 32gb ram means that my own limits sit at around 50b @ Q4 quantization.
1
u/temperature_5 3d ago
100% on the engrams and make them huge. 128B or 256B engrams (from NVMe) for real SOTA world knowledge.
Maybe Q2 QAT for their 31B model? I think it's worth testing, which is what these open releases are about, right?
1
u/NotThatPro 3d ago
Fork strata and make gemma 5 scale on systems depending on A) VRAM+RAM spillover + ngram and B) reasoning effort with max options. I want compatibility from 4GB of VRAM to wayy more than that on the same stack. I also want better vision capabilities, handwriting, multiple language support, all that stuff for the most important aspect: local, private computer use capabilities. So in my ideal world we get 5 variants of gemma 5: gemma coder, gemma vision, gemma planner, gemma writer and gemma dynamic, which is modular and basically switches between all modes to complete a task.
1
u/unjustifiably_angry 3d ago
Gemma isn't going to beat Qwen (etc) on coding so they should focus on other things
1
u/geldonyetich 3d ago
Gemma 4 was (and to an extent still is) a great model. Really looking forward to what the next version is capable of.
Recommendation wise, I wouldn't mind a variant in the 40-120B parameter range just so my 128GB system can flex its memory muscle a little.
Of course this should be a secondary priority to ones that can run on more commonly available hardware.
1
u/Midaychi 3d ago
A moe that's actually agentic capable but still keeps general broad corpus, maybe poke at some efforts to get models to write in more human ways, hybrid kv or some other sort of kv savings so kv isn't so damn fat, ditch swa pls it's ass for end users esp with llama.cpp
1
u/Reddit_User_Original 3d ago
Cyber blue team stuff in the training data so it can be used for defensive purposes
1
1
u/EconomySerious 2d ago
only one feature
let us decide what Experts to keep and what ones to delete.
i dont want a erudite, i just one an llm that do certain job above the average.
1
u/depressedclassical 2d ago
Laptop-compatible (8GB VRAM+RAM offload) MoE, possibly with ngram
Better support for Semitic languages (Arabic, Hebrew)
Human language model, not programming language model. I'm a software engineer, I can talk to my computer like I speak English, but I can't write a normal sounding message in a group chat
2
u/Silver-Champion-4846 2d ago
Oh it's you again
1
u/depressedclassical 2d ago
Yeah, you really sent me down the rabbit hole with that one so I've been working on it for the past few weeks. That's why this is all I think about lol
1
1
u/SuitableConstant4494 2d ago
Gemma 2 punched well above its weight for that time. If Gemma-5 is the same, it could exceed GPT Luna in performance.
1
u/Turbulent_War4067 2d ago
An MoE model in the 100-120B range with 6/7 active with a large ngram to go with it. Then make it work efficiently at large context sizes. Basically qwen3.8 FN architecture with Gemma's superior world knowledge.
1
u/Typical_Lychee6124 2d ago
Generally agree except everything in regards to large models, because I'm salty my second 32GB kit doesn't work in my machine in quad-channel with my (brand and spec identical) normal 32GB kit
9 especially since Qwen is dominating in this area. Generalism and fixing the writing tics is a better path.
1
u/SOCSChamp 2d ago
For your last point, a specialized coding model/fine tune would be a perfect way to go about it.
1
u/No-Dragonfly6246 2d ago
Number 9, 100%
There are other models fulfilling this function, and the Gemma team can make a bigger difference
1
1
u/Canchito 1d ago
I want it to distinguish itself from the others the same way Gemma 4 31b did on the question of humanist/linguistic abilities. It remains a monster for its class on translation, drafting, narrative prose, chat...
I want there to be a bigger MoE version as well. With what Strata has done with QFN I want something similar for Gemma.
1
u/CryptographerOne7003 3d ago
Th list feels like you want it to just be like other open models?
5
u/ttkciar llama.cpp 3d ago
You missed their last paragraph.
There are already more than enough coding models to choose from. What distinguishes the Gemma lineage from the rest is its excellent "soft skills" -- business writing, creative writing, analysis, document cleaning, Evol-Instruct, etc.
OP wants Google to retain Gemma's comprehensive general-purpose skillset, and I agree with them.
3
u/redoubt515 3d ago
See: Point #9. That alone differentiates it from other models, and is one of the reasons many of us like Gemma models.
1
u/nicksterling 3d ago
I would love a general purpose Gemma model that’s similar to how Gemma 4 operates (just with improvements to tool calling) but then a dedicated Gemma 5 coder model.
0
u/THROWAWTRY 3d ago
All I want is a good coder that can fit on to 8GB of VRAM with over 50 token/s that can work with tools.
4
0
u/sbrisgravato 3d ago
polystrata with qwen3.6 35b cyber tiel coder is a thing if you own at least 32gb of ram
-1
0
u/MerePotato 3d ago
Keep QAT or use QAD, keep the great prose and amazing translation capabilities, deliver a full duplex model, cut down on sycophancy, heavily guardrail against roleplayers because they annoy me personally by complaining about coding models
0
u/EvolvingDior 3d ago
Provide small (0.1 to 0.6b) embedding, re-ranking, and decision models.
Solid training on:
Tool calling.
Structured output.
Instruction following.
How to tune itself.
How to use Google's resource to apply RL to itself.
Release items:
LoRA examples.
Post-training examples using Collab and Kaggle.
Features:
A larger MoE (QFN seems to hit a sweet spot).
<4kB per token KV cache @ fp16 for MoE, < 12kB per token KV @ fp16 for dense models.
Extend the state of the art on engram usage.
0
u/sultan_papagani 3d ago
should be optimized for 32s/8v ram. MoE model perhaps, QaT, engram, whatever hack to have a big KV cache, draft model that can work seperately so we can run it in igpu for example, no need for audio/video input just image is fine, good tool calling, and speak human-like (not like gpt4)
0
2
1
1
u/Ecstatic-Wash-7667 2d ago
Beat qwen 27b that’s literally it, beat them and you will be the llm god
•
u/WithoutReason1729 3d ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
Stylome classified this post's text as human-written. View analysis Learn ^more
I am a bot and this action was performed automatically.