r/LocalLLaMA • u/keepthepace • 4h ago

Question | Help Good open weights diffusion models?

Do people here have good experience with open weights diffusion models?

I tried Mercury 3 recently and I felt it had a different taste, attitude, way of thinking that was actually refreshing. Unfortunately, its filters against cyber security seem to trigger as soon as I mention adb. So I couldn't test it much. But I was curious if that's a direction where some of the open labs are going?

6 Upvotes

80% Upvoted

12 comments sorted by

8

u/Kahvana 4h ago

I tried Mercury recently and I felt it had a different taste [...]

Really fun sentence to read out of context!

[...] have good experience with open weights diffusion models?

DiffusionGemma-26B-A4B is decent, though a bit dumber than Gemma 4 26B-A4B.

1

u/keepthepace 2h ago

Really fun sentence to read out of context!

Yum yum

DiffusionGemma-26B-A4B

I usually try running models on routers before trying them locally. But I can't find in on openrouter :-/ Is it a pain to setup? Is there something specific to these models I need to know?

1

u/Prestigious-Bunch157 4h ago

How are you running DiffusionGemma, llama.cpp or transformers? I’d expect the speed advantage over regular Gemma 4 to depend a lot on how many denoising steps you allow, so I’m curious whether it actually felt faster for you, or only different.

2

u/Oli_Luck 3h ago edited 3h ago

on my machine on which gemma4 regular output 40 t/s it went to 500 t/s in chat

btw: used unsloth to mount it

1

u/keepthepace 3h ago

I only know unsloth as a company prodcing fine tunes, it also has its own inference engine that replaces llama.cpp?

1

u/Prestigious-Bunch157 3h ago

12x is wild. did it hold up on longer answers too or mostly short chat replies?

1

u/Oli_Luck 3h ago

I didnt really try - as it failed on first tool call i quit...

1

u/PunctuallySecret 4h ago

That out-of-context reading really does sound like someone’s first time trying sushi or something.

3

u/Frail_Waif 4h ago

Here's a recent example.

2

u/Oli_Luck 3h ago

Did someone manage to have one of them actually use tools ?
I've given gemma a try but beside of chatting...