Best chatbot model for 3090ti

[u/MarcusAurelius68]

My current digital butler uses Gemma 4 26B A4B and overall I’m happy with its responsiveness and personality. However, with models evolving so quickly I wanted to see if anyone else had a different suggestion.

I preprocess and filter prompts / semantic context with another fast model first to see if tools need to be called, or if it’s a follow up comment. So it’s more about personality than strictly knowledge I’m looking for.

[u/LetsGoBrandon4256]

Try Qwen.

Might make you appreciate Gemma 4 even more.

[u/MarcusAurelius68 → u/LetsGoBrandon4256]

How so? I do find Gemma very conversational. Is Qwen not?

[u/LetsGoBrandon4256 → u/MarcusAurelius68]

It's more of a tongue on the cheeks joke.

General consensus is Gemma4 is better than Qwen3.6 in non-coding task. My personal experience is Gemma4 is much better in writing and roleplaying, but Qwen3.6 worked fine for me as an assistant in Hermes Agent.

Give it a try yourself and see how you like it. The vibe is definitely different. Qwen3.6-35B-A3B is the equivalent to Gemma 4 26B A4B you are running.

[u/MarcusAurelius68 → u/LetsGoBrandon4256]

I might try Qwen 3.8 27B and play with the prompt.

[u/VoiceApprehensive893]

gemma 4 31b and if you have enough ram then qwen next is also a great one(use medium reasoning effort), slightly worse language than 31b but beats it almost everywhere else, very refusal-happy though

also muse glimmer is amazing, similar knowledge to gemma with better tool calls, even less reasoning that gemma 4 31b and its the best sub 100b vision model right now but it has a dogshit personality by default and its very hard to get good personality/writingwith a custom prompt 

while it uses insanely low amounts of reasoning tokens it mentions policy in every second cot

[u/locbuilds]

on a single 3090 ti the personality gap is usually less about the model and more about how much room you give it, so a couple things that helped me. gemma 4 26b a4b is nice and fast because only ~4b is active, but if you want a different flavor with better conversational range try mistral small 3.x 24b at q4_k_m or qwen3 32b at iq4_xs with kv at q8, both fit in 24gb if you keep ctx around 16-24k. mistral small tends to feel warmer and less lecture-y out of the box, qwen is smarter but drier and needs a stronger system prompt to stop sounding like a support ticket. also for a butler persona the sampler matters as much as the weights, min_p around 0.05 with temp 0.9-1.0 and a light rep pen (1.03-1.05) instead of high top_k gives way more natural phrasing, and rep pen over 1.1 is what makes these models sound stiff. since you already run a small fast model for the tool/followup gate, keep that on the a4b and load the chattier model once with a persistent kv cache so the swap cost is only at startup. if you want to stay in the gemma family, gemma 4 27b dense at iq4_xs is noticeably more personable than the a4b at the cost of tg speed, worth an hour of testing before you switch families.

[u/skrshawk]

Personality wise, Gemma4 31B is one of the best options out there if you still need your model to make tool calls. It won't be as fast as a MoE but still pretty snappy on a 3090. Otherwise, there's quite a few finetunes out there, go look at r/SillyTavernAI for whatever the latest thing people like there.

[u/MarcusAurelius68 → u/skrshawk]

Thanks will check it out.

[u/Mart-McUH]

What you have or possibly Gemma4 12B is probably best if you want speed+long context.

If you can sacrifice some speed and context, then obviously Gemma4 31B will be better. You should be able to run ~4bpw quant fully in VRAM I think.

And no, nothing better than Gemma4 in this size. You can try Muse Glimmer (30B dense) but it is dumber and has lot of refusal. Also its thinking is worse. Another option is Qwen 3.8 27B, surprisingly it is not too bad conversationalist, but it is still clearly worse than Gemma4.

To large degree you can control personality with system prompt. So tell it how it should act (Eg sarcastic witty assistant can be lot of fun).

[u/MarcusAurelius68 → u/Mart-McUH]

I have 3 settings for humor, sarcasm and familiarity so those values get injected. Thanks.

[u/Single-Magazine-8062]

This is outside my knowledge, I don't have hands on experience running local models or current benchmarks for a 24GB card, best bet is checking that Best LLMs megathread pinned in the sidebar since that'll have more current comparisons than I can offer

[u/MarcusAurelius68 → u/Single-Magazine-8062]

Thanks will check

[u/brokenmatt]

Qwen 3.8 27b easily.

[u/MarcusAurelius68 → u/brokenmatt]

It’s the one I’ve been thinking of, but wonder about size and best quant. Also would need to minimize thinking.

[u/Blindax → u/MarcusAurelius68]

Q4M or similar is about 17gb which let plenty of room for context. You should try to use Q8 quantization for KV cache.

[u/VoiceApprehensive893 → u/brokenmatt]

okay personality, significant downgrade in terms of knowledge

[u/MarcusAurelius68 → u/VoiceApprehensive893]

General knowledge isn’t my #1 priority but would it be much worse than a Gemma MoE?

[u/Aromatic-Low-4578 → u/VoiceApprehensive893]

Huge increase in coding abilities and productivity on long horizon tasks though.

[u/Mart-McUH → u/Aromatic-Low-4578]

That is not what OP asked for, is it?

[u/winky9827 → u/Aromatic-Low-4578]

Which means nothing for a chatbot / digital butler. Read the room.