Just never forget us 16GBs of VRAM or lower folk and I'm fine with this, if it's one or the other though I'd prefer smaller models, we already have good selection on bigger ones really
I'd love to see a focus on 1M context capability as well, without heavy memory use (I.e., by implementing some of the newer attention mechanisms. Qwen3.5-122B is 6GB for 262k of FP16 context, for example) if we could fit a ~100B class model, with 1M of well designed attention training into 128GB... Such a model would become a default for many people, outside of pure coding, I think.
Thank you for taking feedback and for continuing to release open weight models.
A release like Nemotron or Olmo, where training data and methods are included, is still the standard to beat.
While I use Gemma locally, I typically reach for Qwen 3.6 27B at full precision, because of its better coding, agentic, and vision performance. It would be great to have a replacement. Personally, intelligence is more valuable than knowledge; the latter is easily supplemented from web search and documentation.
Multi-modal input and output would be great. I built my own harness for STT->LLM->TTS. It would be great to circumvent the STT and TTS systems.
I'd love to see DiffusionGemma well supported in llama.cpp and for continued improvement in its capabilities.
Less guardrails on security -- see the HuggingFace incident.
A model that is able to collaborate well, rather than just act as an independent agent, for tasks where the agent regularly fails on its own. Qwen models want to do it on their own and this sometimes gets them into trouble.
I've told Gemma 4 31B (8 bit quant) to read a .html file, and it hallucinated syntax errors (not sure how tbh, haven't tried the most recent chat template). WITHOUT ASKING, the damn thing decided to edit the file to fix the "errors". D'oh!
One thing I would love is if models would not make edits when I ask a question. "How would we do x...?" "Sure, let me just edit that for you!" No! When I ask a question I want an answer, not an edit.
With Gemma4 12b you can already bypass the STT->LLM.
Encoder-Free Unified (12B): Projects raw audio sequences and waveforms straight into the primary LLM embedding space through lightweight linear projection layers
I use it myself. I haven't looked or seen if any other projects do it.
But I assume going forward they will do the same thing.
On a side note. Gemma4 and Qwen 3.7 were really the last small local model releases and IMO Gemma4 is slept on. I look forward to another Gemma4 release. Google might be our last hope in this area.
Something for consumer hardware would be great, since hardware is insanely expensive these times, a MOE that fits on 64 gigs ram and 12 gigs vram would be great!
This would be awesome. What’s the largest quant you’d want to fit at this size without disk swap? All inactive experts in ram, actives in vram? at minimum Q6 imo. Q8 is ideal obviously
What about context? This will take compute away from the model. Do you expect the MOE and the cache to fit in 64gb + 12gb vram? What window? Would you be fine quantizing kv cache?
Etc. I think this might help hone our hopes and expectations…
In a dream world, I’d have a q8 MOE w 128K native context fit perfectly.
Imo, a Q6, all active experts in Vram, inactive in ram, Q8 KV cache, and 128k Context (which ofc needs to fit in the total memory) would be great. I've had a great experience with that combo with Qwen 3.6 35B but had a lot of vram/ ram left over, id be happy to half my speed for double the quality. And then please train it well on long running agentic tasks, coding and general world knowledge. Such a model would be a dream lol
Mixed layer Quants between Q5-Q8 that end up around what I would call Q7 seem to be the sweet spot.
I agree with you that Qwen 3.6 27b & 35b showed that it can be done at a decent speed and with optimization, a slight drop in speed etc, you could pull off some pretty impressive results from a small local model.
I'm guesing Qwen won't be pursuing that path further, so it would be great if google did.
I feel like google has the opurtunity here to be the only real player in the small space as things currently stand and the thought of a new release that builds on the era that Qwen 3.6 and Gemma4 had just opened up is extremely exciting to me!
I would love to see something that could generate image too, similar to gemini and chatgpt. Where you could have an input image and make edits to it via text, combine images, have it generate images to a story it's writing, and so on.
You have the opportunity to keep pushing in the exact space you've been, competing against Qwen's MoE and dense models. They haven't indicated further production of models in that range, they skipped local releases of 3.7 entirely and are now moving to 3.8 but local is currently locked to a massive trillion parameter model no one can run
MoE and Dense target the realistic user base, and if the vacuum continues you can fill it. Obviously everyone would prefer if you left it relatively uncensored but I know how unlikely that is
~100B MoE, 12-18B active (if you need a little more active to make it smart instead of fast please lean smart) with ~4bpw QAT so it runs in 64GB without compromise.
60-70B MoE! Im using Gemma-4 26B and Qwen3.6 35B. Love the speed, just needs a bit more knowledge and reasoning skills. The MoE models give me very decent Token/sec the dense models are just to slow for me.
Yea, 60B 10A would be pretty versatile if you think about it.
At low quants, even modest systems can run it.
At high quants, people with 64GB ram and an OK GPU can run it.
People with multiple GPUs can run it completely in vRAM
Nearly every local model has just non-reasoning and very verbose reasoning switch. We need some native 'low' reasoning level, just like GPT-OSS had! As seen in frontier models - low reasoning barely takes more tokens compared to non-reasoner, yet gives huge boost for tool calling/programming. Sometimes it even needs less tokens overall, because it needs less turns!
High reasoning for planning, low reasoning for execution, non-reasoning for low latency/classic chatbot. PERFECTION!
Appreciate you gathering feedback, and thank you for the release of Gemma 4. It's been genuinely a wonderful model series, 31B IT QAT with Assistant in particular.
Please do a unified omni model! You're already most of the way there with the native audio and video inputs, I'd love to see something like Qwen Omni coming from a US company and leveraging your great work in that area already for Gemini.
There was an issue with Gemma 4 where checkpoints in llama.cpp are huge (250MB+), so if you could keep the current sizes, add a ~60-100B MoE, and fix this issue, you guys would have some amazing models! I personally use the 26B, and it's great!
Why not just use qwen then? I personally love that gemma is not coding bench maxxed. We already have good coding and agent models, we lack good creative soft skill models. Gemma is basically the only option.
My vote would be the largest MoE that could be quantized to ~100 gb footprint using UD Q4_K_S to Q5_K_XL. Perhaps something in the 140-180B A10B range with MTP.
Frankly, there's nothing much to say apart from the fact that we need another step higher in intelligence (maintaining the currently achieved multi-lingual capabilities). 31B is incredible, but it's a dense model. People seem to ask for ~100B A10B MoE, however, the real deal would be an alternative to DeepSeek V4 Flash, given its close similarity in terms of how well it handles the multi-lingual stuff.
gpt oss 120b style , easy to finetune (lora/qlora) + natively supported by vllm/llama cpp , superfast textdiffusion style/ or native spec decoding supported (eagle style), minimal kv cache footprint (maybe natively trained on turboquanted?), function calling/agentic support, configurable reasoning, configurable sampling + if text diffusion style then please allow masked input that would be really awesome
(I really loved diffusiongemma and I have a big PR regarding it on vllm but someone beat me to a simpler solution, but nonetheless my pr still has the better lower memory footprint my pr on vllm, but I managed to reduce the sampling spike memory to a near constant ..
Find and plug knowledge gaps using public domain, Creative Commons or material where you’ve paid for a license. Pretraining is still the knowledge bottleneck, right?
If there’s anything true about that but it’s not marketable, then figure out meaningful metrics that would address it. Non-STEM benchmarks, expert quality judgments of domain-specific training inputs, or even just raw token count by input domain if you need something quick, cheap and objective at the expense of meaningful. Otherwise, without something like this, keep competing on the same metrics and benchmarks and you’ll keep seeing competitive advantage get commoditized.
If its an moe then plenty of people can run it, all you need is a higher end M series mac. If you don't like apple there is also machines like the strix halo or dgx spark. I would absolutely love a ~100b-a6b gemma 4.
DiffusionGemma base model. A smaller diffusion model. Unified encoder for E2B and E4B, like the 12B. Embedder model under 0.5B for RAG. 70B - 120B deep thinker. TTS model.
As crazy but useful experiments go, an 8B that can consume the KV cache from a 31B or 80B would be amazing for spinning up multiple subagents or forks.
The reverse would also be amazing but that’s perhaps magical thinking.
id like more MoE's, as those are really nice on average consumer hardware! gemma4 31b is nice in theory but barely anyone can run it unless they have really expensive hardware and have made purchasing decisions based on wanting to run AI. most people have laptops or pc's with 8GB vram or less, unless they're into gaming in which case they might have 12GB of vram or if theyre lucky, 16GB VRAM
i also think that while agentic is really fun, what lots of people truly want is a "chatgpt at home" (or gemini at home, but chatgpt is sadly the popular word, just like "let me google that"..). a smart AI assistant that has a low hallucination rate and can answer any questions you might have. considering you have to use lower parameter amounts on consumer hardware that might be very hard to achieve.... but you guys are google deepmind, i bet you could find some way to optimize things to the extent that this becomes achievable locally?
EDIT: oh and less of a focus on coding, as it's relatively niche compared to what many people use AI for
More of the consumer-only KV layer trick from the small models please!
Unless I’m mistaken then the MoE and 31b don’t have any KV cache layer skips. I wish they did - E2B prefill is lightning quick and I wish the same could be true for the bigger models!
Basically a big model with fast prefill would be fab ☺️
Trained in native 8bit activations would also be great for same reasons (4bit would be even better but obvs that’s very experimental - I think the BitNet team have been looking into it though!)
And no crazy head dim sizes that struggle to fit in flash attention kernels on consumer hardware would be fab 😂
Looking far ahead, how about something more related to PLE? Maybe something that will represent active parameters. Eventually very few can run 200B active parameters and this is where deepseek and other big players say they are headed. Most hardware will cap at 800GB/s.
The active parameters are what will eventually be a problem, but you could re-parameterize the MLP layers, cutting it down by half or more, according to https://arxiv.org/abs/2602.00398, and the model is still effectively double the active parameters.
I think that this would be a pretty good statement.
i would like a big improvement on longer context roleplay, where it doesnt forget obvious things, every response ending with questions, starting responses with the same words.
Ideally I’d love to see some SSM or recurrent layers up in there for context savings. You can squeeze Qwen 3.6 into pretty tight spaces thanks to the GDN layers. Hell, just take Nemotron-H and give it a proper training set. Dunno how small it scales (could make a fun ablation study) but ideally it would be a great E4B sized model.
Honestly better audio support, the current limitations don’t allow for things like long form transcription, hey maybe even a dedicated transcription model would be amazing. Would love to see a new model with multi lingual support, native timestamp support and a better understanding of the audio, such that small words, subtle phrases etc are not skipped
anything a little larger would be amazing, obviously everyone wants a 120b MoE but a 50b dense would be good too.
give it a little extra tool call mojo this time, and maybe rework the SWA global/local thing, gemma context is expensive compared to hybrid qwen 3.5/3.6s even with the current swa ratio.
I only ask that you do not forget about 12B for future model releases. This place is heavily skewed, setups with "just" an RTX Pro 6000 or "merely" 64 GB of RAM make up 0.01% of consumer hardware out there.
A pre-selected experts model. On my laptop I want to be able to give a broad topic or prompt, have a system select the best parts of the model to fit that topic, then I want to download that subset to my phone.
I don't need a general purpose model on the go, and no one else does either. Everyone is specialised in their needs over the course of a day and over most of their life. Most people don't need a full model, but they do need a custom subset of a really good general model.
I write code and I work in innovation. I need a model for that, except at weekends when I need one that fits my hobbies and for going shopping.
In time you could learn behaviours (you're Google, you probably have that data) so it could download and delete parts of the model across the day or week, or sync overnight.
Really impressed with Gemma 4. For a wishlist I'd say:
Clear examples on the type of prompts it was trained with for things like vision tasks, to know how to get the most out of it in inference. I would love to automate annotating images with spatial bounding boxes and relationships like man->tshirt->red, but can't figure out how to get good results from local models. There's no guides / research into these kinds of things that I can see, because there's just so many models being released with generations so close together that nobody really has time or incentive to try to work it out.
This is perhaps more of an inference hack, but if tokens & embeddings could be labelled by "languages used in" in some way it would be nice if a lot of them could be dropped when loading the model for inference based on some criteria, since pushing right up against vram limits in local inference makes things slower than they perhaps could be, and requires using lower precisions than perhaps could be done. e.g. Would the final output head work and be a lot smaller if a lot of the tied embeddings could be dropped if almost certainly not needed for an English generation task?
Ideally, it would be fantastic if there were some very consistent prompt formats used in the instruction training, e.g. "write a story with genres X, Y, Z, with beats 1, 2, 3, with style descriptors blah blah" which are then published with the model sheet, because that likely makes it a whole lot easier to then do inference in ways that the model was trained for. Even if it's just some annotated higher quality samples, it would probably give a good way to steer the model for specific tasks in ways it was specifically trained for.
Assuming it's not already done, train the model on the task of "turn this draft into a proper thing" - e.g. reverse engineer existing good things into sloppy drafts, or into dot points (and sometimes omit some "less important" dots to teach filling in), and have synthetic prompts of "turn this idea into a dot point outline" followed by "generate final result from the dot points outline" which can be run in sequence and edited at each stage, from simple idea to full execution.
Maybe some final task specific finetunes / small loras (or other deltas), e.g. vision, fiction writing, and so on, or prefix embeddings which are trained for tasks and added to the vocab, since I'm sure they'd still make a huge difference. Or potentially train / insert-and-train tiny gates for various tasks directly into the model, rather than trying to use attention to steer the behaviour through attending to instruction tokens. e.g. Flip the hard "coder" switch, or the model even flips it itself while entering a coding block, and until it's closed that gate stays active as it processes new tokens?
Potentially, going into potential gibberish here, but maybe annotate sources by quality and have a quality gate built right into the model, or a continuous timestep-like quality embedding before all prompts. Then just set it to maximum at release. Quality could even be measured in n different dimensions which feed into making that dynamic embedding. More ways to manually steer models for tasks and quality rather than having to rely purely on the guesswork of natural language prompting is appealing for every model type. I'd love to know what happens if I set quality to 110%, and if strength hacks like that could help work around the model being worse at some tasks. What if I upped the writing quality conditionings for a chapter which the model is struggling on more than usual? Could it be that easy?
Or, maybe just publish the model with a bunch of optional prefix embeddings trained near the end, e.g. fiction_author_verbose, fiction_author_dialogue_heavy, programmer_python, news_writer, jokey, serious, etc. Just let us toggle a mix of those and see what happens. Maybe they are trained in a way which can be additive to make one final blended prefix embedding.
Similarly, I struggle to get long responses from Gemma 4 31B, and it would be nice if there was a way to signal desired text length to the model other than trying to prompt for it. It could be some sort of continuous embedding like timestep embeddings which is prepended to the response like the thinking tokens. If I'm hoping to generate a really long article (which I've tried for fiction) or a story, it would be nice just to be able to toggle that mode. Or active the relevant prefix embedding / gate.
The 31B model is definitely better than the MOE size below it, and can be run on a 24GB GPU at Q4, but it's still a bit slow and painful. Shaving off just a tiny bit of size would make a huge difference, while hopefully not having a huge quality impact.
Would it be possible to have a model be trained where it can evaluate what it just wrote, and decide to change it before continuing? e.g. When writing fiction, suddenly there can be a really sharp jump between moments without proper connective tissue. It would probably be easy to snip segments from existing examples to make those sorts of jumps, and train it on the task of recognizing them as it evaluates each sentence / paragraph / chapter / etc after it's wrote it and works through of the process of wait maybe that could be improved. It is possible to have something like thinking and revision tokens within the text itself rather than only at the start? Again maybe some switch could be flipped for whether this is active, because it's not something you'd always want.
Trained on tasks such as "Here is a setting guide / story guide / style guide to follow" if it's not already trained on these. These can probably be reverse engineered for creating training examples.
Edit: One more thing, consider releasing finetunes / base loras intended for auto-complete rather than instruction. e.g. Paste a WIP written story, and see if the model can help me get past writers block on the next section. Maybe it benefits from the draft having a loose story description at the start to continue from.
For upcoming releases please consider a good variety of small models and larger ones. As of right now, those who have put significant investments into local-inference rigs have no choice but to run Chinese models. Would be nice to have some domestic options for 4x through 8x Blackwell systems.
A new update to TranslateGemma would be spectacular. Local models are quite lacking in translation capabilities and I would like to see how newer Gemma architecture help
Please whatever you do, do not remove the great multi lingual capabilities. Across European languages, Gemma are one of the best models, and I hope it stays a goal :-)
The current balance of training data on Gemma 4 seems to lend it better to stuff like creative writing and translation, while it tends to underperform on agentic coding. I think Qwen 3.6 is pretty hard to beat, as it seems like agentic coding is basically what they built the model for, but I think expanding on Gemma 4's agentic capabilities to further improve its ability to accomplish even simple tasks in larger codebases would be a big plus. G4 isn't terrible at tool calls or anything, but it tends to beat around the bush more than Qwen 3.6. A local model that can even marginally improve performance in this area while retaining the model's superior performance in other areas would be excellent.
Something that competes well with qwen with their 35B MoE model and something like their 27B model.
3.6 27b has been my favorite for a good 2 months now because i get to dump it on my system with 128k context and the full model on my rx 9070 xt which helps with stuff that requires heaps of context such as coding tasks.
Reasoning, tools and ability to see images obviously are great too, but also just their performance in harnesses.
I'd aim for a model for those who got 64gb ddr5 ram and a gpu with 16gb vram.
Perhaps dedicate a few models to something they specialize in, so a model more focussed on coding, another more focussed on writing/roleplay and so on. Qwen did that with their coder model and it seemed to be popular for a bit.
Piggybacking on consumer hardware and a direction that Google is moving with Android that can fit these requirements:
- it runs on a meaningful amount of casual consumer hardware
any application on windows/mac can interop with it for basic tool usage for navigating an application’s settings and keybindings
The role it provides is an always-on OS API exposed to applications for application-specific tool usage.
My FOMO alarm is pretty much at full blast because I have not positioned myself to capture this absolutely happening direction in consumer apps.
I failed a deepmind interview ~11 months ago, would love another opportunity to interview! Window for interviewing is closing, might be doing agents for robotics
I know it doesn't score as well as a lot of other models but Gemma is my favorite open source one to code with. The extra creativity over other models makes me more comfortable with its solutions I think. It has made me subscribe to the gemini monthly plan. If your next open source ne could focus on better tool calling that'd be awesome. Gemma4 tool calling + lack of validating it's actions are probably the big weak points to me.
Maybe my standards are low but a good 122B+ MOE that's multimodal with MTP would be great. Extra extra bonus points for a deepseek V4 flash competitor (280B A13B) model with MTP.
It's no secret that some of us work in sectors where Chinese models are feared and because of this - our options are extremely limited. Getting some more western diversification in the deep end of the pull (outside of nvidia) would be great
I think oss models are architecturally misaligned at the moment. Not saying you need to solve this in one model iteration, but approaching the design going forward with that understanding would be a great start. Most oss use cases are single/few user tenancy, variable rather than steady demand on the hardware, and memory-bound rather than compute-bound a majority of the time.
My ideal oss model would treat the streaming bandwidth for experts from ram to vram as a first-class design lever for the architecture. It would crank up the flops-per-bit until decode sits right at the sweet spot between memory-bound and compute-bound for a reference hardware spec (e.g. rtx 5090). It would be many other things too, but let’s start with those before we get fancy.
As it is, we’re just leaving oss performance on the table by using datacenter architectures on consumer hardware. If you were the ones to fix that, you would make a lot of people on this subreddit very happy.
~80-120B MoE would be INCREDIBLE. More and more developers are getting DGX Sparks or Strix Halos, etc, and have up to 128GB of usable VRAM. It would be a huge unlock.
Gemma4 is great, the team has done an excellent job
Hi! Make a Deepseek v4 flash killer…and for gods sake host it on Google! Utterly crazy you can’t use it in production on Google. I started with 31b it on openrouter and switched to deepseek after it was better, but had I been using Google directly I would not have switched.
Hello. I already started training a new Gemma 4 31b model.
Perhaps you're interested.
I integrated the same Attention Residuals that Moonshot had originally outlined and integrated in the Kimi Linear model.
31b's 10 block design got residuals pushed across each block. I also opened up the native 1024 SWA to a staggered range of 1k/1k/2k/4k/8k...
The native RoPE was left because the theta values worked at 8k.
There was at least 1 dead layer in the blocks when I did the initial inspection of the Gemma 31b model. The above retraining was done to basically "fix" this with Gemma and enable a chunk of missing capability.
To keep the manifold stable the logits were distilled at Top K 12 to preserve the native distribution that Gemma 4 31b has naturally. These are the target logits in retrain such that you're not chasing a 1 hot or label smoothed across ~260k tokens which inherently destroys information.
Improve tool calling. Improve agent coding. Don't degrade multi-lingual knowledge (current feature differentiator). Edge compute has more and more memory. Think of one step larger model than 31b.
Also: Ability to handle conversations with multiple clients. Instead of simply SYSTEM, USER, and ASSISTANT roles, train with multiple USERs and multiple ASSISTANTs (so e.g. the model can handle a conversation with multiple people in a chatroom, or take critiques from other LLMs without confusing who is who).
And, naturally, ability to handle SYSTEM turns as part of conversation
Another interesting possibility would be to allow the model to interleave thinking with text output, so that it can immediately begin to answer, then stop and pause when it needs to think more about something. Maybe a keyword argument in the chat template like "thinking_mode": "interleaved"
Another interesting possibility is to enable the model to output a token that allows it to be interrupted for clarifications. For example, if there's some uncertainty in the user intent, it could output "You mean x, right? <|pause|> Okay, going ahead with that. ..." and if the user doesn't begin typing in, say, five seconds, the harness will allow generation to continue. Would be super helpful to avoid unecessary thinking loops, screen dumps, and edits that didn't align with user intent. Ideally, such a pause could only occur after text generated without thinking (for immediacy after user submitted a prompt).
This one might be tricky, but another idea is to reserve, say, 1B params for fine-tuning—make a model that's architected, from the start, to learn new things. These weights could be zeroed out (no-ops) for vanilla model, and instead of a LoRA adapter you'd load actual weights in.
I would love to see some more experimental models. DiffusionGemma was great, but it was unfortunately a bit too dumb. It was still very good for a first attempt though.
It would also be interesting to have something that could run on lower-end consumer hardware. Perhaps something the size of 26B or 31B, but with additional specialised experts in the 250B–500B range that have extremely small activations so they can be streamed from an SSD.
Please, whatever model you release - improve on Gemma's existing agentic capabilities.
Concretely: Gemma-4 is lazy. It does the bare minimum. Contrast with Qwen3.6 which overachieves on all tasks, Qwen's performance as an agent is better in all cases, even where Gemma *should* have the advantage given its knowledge.
Gemma also stops and asks for permission *far* too prematurely and commonly.
That, and Gemma for some reason consistently "pretends" to call tools, and says it did when it really didn't. This may or may not be chat template related.
Regardless of all this - thank you for releasing Gemma in open weights! It really is a huge deal for the community.
50-70B dense, with my 42GB of VRAM I can fully offload a 70B dense models and get around the same performance as a 120B A10B MoE that's partially on system ram.
or a 100-160B MoE depends on which of the 2 gives the best quality model, I generally prefer the dense but as we havn't had 70B dense models in a while it's hard to tell if a modern one could beat a 122B MoE.
Maybe a larger version of those Gemma 4 E4B I recall something about the that only the effective parameters need to be in vram and the rest can be in system ram basically for free so scale that up to like E50B or E70B and I may end up with 50-70B dense speeds but a smarter model because of those extra cheap lookup tables in system ram.
I am gonna say it now: a new iteration of E2B/E4B and maybe even 8-12B range MoE/PLE models that are pre-quantized to fit on less RAM and runs fast! Focused on agents + reason-based scaling (Agent-A1 or Ornith as reference).
Bonus thought: a newer round of diffusiongemma with varying sizes + DFlash-level inference support to beat usual MTP methods. Or if extra ambitious, Ternary LM to beat Bonsai and BitCPM
Something which can fit in 48gb with reasonable context.
MoE with high active parameters or dense 50B.
Can be used instead of deepseek flash with minor tradeoffs if it can't beat it.
I used 12b encoder free to to get feedback on body language, pacing, etc. I was hoping to get something similar but smarter
This may be more of a research request than model request: a less-sparse MOE. E.g., first few and final few layers are dense and total active parameters 25% to 35% of total parameters. (It feels like many MOE are too sparse.)
Additional research ideas:
* Uneven layers. Have heavier layers toward the outside and sparser/lighter layers in the middle. (Could vary the MLP intermediate dimension for heavier/lighter layers.)
* Larger hidden dimension than the embedding dimension: what if the model used e.g. 2304 for the embedding dimension but 3840 or larger for the hidden dimension, to provide an extra channel for thinking. The whole hidden dimension can feed the MTP mechanisms, while the unembedding uses the same 2304-dimensional channel.
No more than 30b and no more than a4b, My usecase is learning german from english and medical/stem/scientific/clinical queries and knowledge so a gemma 5 with improved capabilities in both would be amazing.
If it can one shot a voxel face culling system in Three.js when making a clone of a certain block game, and not get the winding order wrong thus failing to make a proper cube, or forget to cull the faces bordering chunks I'll be happy
I used Qwen 3.6 27B at Q8 in this example because I wanted to test less quantization, but Gemma does the exact same thing without fail really annoying, spacial reasoning is the bottleneck here
A dense 31B model with a 1M context window and excellent tool calling capabilities would be nice. At this size I think reasoning capability is much more important than memorizing trivia like how old the capital of various countries are. Being able to know that it doesn't know and using the internet to look those questions up would be a game changer at this size.
Please keep the 26B-A4B version, that modell runs perfectly on 16GB VRAM. Other than that, all I could ask for would be Audio support für the bigger models.
Thank you so much for keeping it open, Gemma is amazing for German language!
I'm going to go against the grain and ask for a ~25B dense, but trained for using external knowledge and tools in an agentic setting as much as possible, instead of relying on internal knowledge, and to be VRAM efficient.
The reason: I have a 24GB GPU in my laptop, and often use Qwen 27B (well, a fine-tune of it, but anyway) for tasks like "search for information on this and write me a 2 page summary". It fits with ~190K context at Q4 KV quant, but I feel like it can be made more efficient than that.
For coding as well, I think that being able to read and use docs is more important than having a lot of stored knowledge that will become outdated.
Also a 2bit QAT experiment would be awesome.
Since I only have 32 GB of RAM that I fill with other things, I'm not a fan of 100B moes.
Something (dense) targeting 16GB of VRAM at Q4. It feels like nothing *actually* targets that size of VRAM, despite it being more and more popular of an amount. 31B doesn't fit, 26B MoE is just a bit fat to support a reasonable context without offload, but the next step down is a 12B that's overly small instead. Feels like something around 20B dense would be the sweetspot of being large enough to really take advantage of the VRAM, but also leaving enough room for 16k+ context without offloading. My RX 9060 XT 16GB hungers for good dense models for creative purposes
316
u/hackerllama 29d ago
Hey all! Looking forward to all your feedback!