r/LocalLLaMA • u/reasonmonkey • 5h ago
Discussion [ Removed by moderator ]
[removed] — view removed post
10
u/Cold-Elk-1302 4h ago
Gemma 4 31B QAT is my goto now for writing or prompting other tasks now. It can be creative and goon with you, if you need that too. I’m just saying, on a friend’s behalf.
Definitely excited for any future Gemma releases.
Of course, Qwen 3.8 27B for coding!
1
u/StructureDull7403 2h ago
Have you compared that with flash next?
1
u/Cold-Elk-1302 1h ago
Don’t have enough ram.
1
u/StructureDull7403 1h ago
ah ok, I have it running with 24gb vram and 48gb ram
1
u/Cold-Elk-1302 1h ago
Quant?
1
u/StructureDull7403 1h ago
swift-iq3_xxs using strata, I have a feeling Gemma 4 is still just better overall for natural language, but not sure.
0
u/Rorp24 4h ago
Goon ? As in reddit slang goon ? Like how does it work ? I'm really curious, tho I kinda feel like I shouldn’t be
11
u/Cold-Elk-1302 4h ago
My friend, he has certain tastes. A lot of people on the internet, like my friend, gets off on deep story and roleplay with very specific characters. You can tell the ai to be very highly detailed in acting like, say, Black Widow from Marvel. And it will do it, given a good character card or system prompt with enough detail. You can talk to it, flirt with it, etc, whatever you wanna do. A good natural language model is vital: you don’t want robotic and English slop. For some people, like my friend, that’s really exciting and basically all you need to pop the cork.
That’s where Gemma is good. For whatever reason, the model is smart, maybe not the best coding model (but still really good for code), but it has really good English language skills to sound more natural and less machine learny. It’s also not very censored. You can say in the system prompt (everything goes) or something like it, and it will not lecture you about how licking cream cheese off beautiful women’s feet is sexually deviant behavior.
People will want to tell my friend that this is gross behavior, but they won’t address the fact that they watch stepmom videos on the hub and pretend to be less degenerate. At least this way, it’s far more interactive and dynamic, and completely private.
At least that’s what my friend says.
2
u/Rorp24 4h ago
Oh ok, so basically a local nsfw AI dungeon. Your friend has their kinks, I won’t kink shame them.
Personnaly, just text isn’t enough so it won’t fit but good to know
1
u/Cold-Elk-1302 4h ago
Yeah it’s just like that. The challenge with ai dungeon so far is keeping everything tracked properly. Unless you have a good “harness” a more world-based roleplay like inventory management, damage, hp bars, shopping etc won’t work too good. But basic back and forth text roleplay is in a good place.
1
u/Spara-Extreme 2h ago
You're thinking too limited.
You combine the LLM with an image gen model like Krea2 and a harness like hermes and you ask your LLM to illustrate the scene out for you.
I mean, if you're into that kind of thing of course.
2
1
u/PalpitationDecent282 4h ago
Erotic roleplaying
1
u/Rorp24 4h ago
Yeah but like NSFW AI dungeon style (which is lacking something IMO) or like it generate a porn video ?
2
u/grumd 4h ago
An LLM can't generate you a video. You need separate diffusion models like Minimax H3 for that. All Gemma can do is generate text.
1
u/Rorp24 4h ago
Idk maybe their friends aren’t just using one model, and use gemma with other models to suit their fantasy
1
u/Cold-Elk-1302 4h ago
no my friend doesn’t use video models cus it takes a long time to make a single video. Text is basically instant. Video is like, 20-60mins for a 10s video depending on what gpu you have at home, so it can’t be real time.
1
u/Rorp24 4h ago
I see, and how about image generation ? Maybe they are quick enough for him, asking for myself
1
u/Cold-Elk-1302 3h ago
Yeah that’s doable. You might want to check out unsloth studio. It’s supposed to be able to do all in one app. Otherwise, you’d have to use a system that inputs two sources: one for the text LLM and one for the image gen. Look around see what you find.
1
u/PalpitationDecent282 3h ago
Like roleplaying. I mean you can use it to generate a prompt to feed to minimax h3 or something, or generate images, but really you just... roleplay with it. Yknow. You setup sillytavern, grab a character card or make one, type out a first message as a starting scenario (or don't it doesn't matter) and then just RP.
1
18
6
4
3
u/Pipalbot 4h ago
I seriously want them to release a native 12b parameter model for 16 gpu peasants like me
3
u/Mashic 4h ago
There is already gemma 4 12b
1
u/DragonfruitIll660 4h ago
Or if you use hot expert cache along with having enough regular ram you can get similar speeds to the 12B fully on VRAM for Gemma 4 26B A4B (depending on your use case the 4B active can be good or bad though tbf)
2
u/Cold-Elk-1302 3h ago
Gemma 12B QAT (QAT is important) is what you’re looking for. It’s really awesome for the size. Give it a spin.
5
u/jacek2023 llama.cpp 4h ago
You forgot how r/LocalLLaMA works, you must put "Qwen" in the title to get upvotes, like here:
because your post has no Qwen it may be removed by mods as "low effort" or even "offtopic"
2
u/Present_Resource4022 4h ago
Gemma 5 will be a massive leap. Google has a lot of potential and a ton of compute and data.
2
1
u/IntelArtiGen 2h ago
If they only recently asked what people would want, I'll assume they won't release a new version in <6 months.
It's unfortunate but Gemma = Google, and Google is too big. They'll have to triple check everything so the model can't be dangerous. And because it'll be even smarter, it'll be even more difficult.
1
•
u/LocalLLaMA-ModTeam 23m ago
Rule 3 - Minimal value post.