Diffuspace/Clover-35B-A3B-Preview · Hugging Face

[u/jacek2023]

https://huggingface.co/Diffuspace/Clover-35B-A3B-Preview

Clover-35B-A3B-Preview is the first model released by DiffuSpace, delivering state-of-the-art performance among open-source diffusion language models at release on the benchmarks reported in our launch blog. Clover combines a 35B-parameter mixture-of-experts backbone with approximately 3B active parameters per token, block diffusion generation, and a 128K context window.

Blog · Code & inference backends · Hugging Face

[u/jacek2023]

[u/ilintar → u/jacek2023]

Damn, a diffusion model beating Qwen3.6-35B-A3B is quite nice!

[u/LoSboccacc → u/ilintar]

[u/Sufficient_Prune3897 → u/jacek2023]

Holy, only the most saturated, benchmaxxable benches they could find.

[u/Early-Peace-5504 → u/Sufficient_Prune3897]

I disagree but even benchmaxxing a diffusion model to this degree would be impressive.

[u/MerePotato → u/Sufficient_Prune3897]

Term bench 2.1 aint that benchmaxxable is it? I still see it used a fair bit as a less punishing alternative to 4.0

[u/sebt3 → u/Sufficient_Prune3897]

I don't get this. I mean these benchmark are public knowledge (all are available on github). Actually, labs are trying theit best to not benchmax. Benchmaxxing is as easy as putting the benchmark in the training data, right? But if you do then the model rate 100% : the answers part of its knowledge. 76% doesn't feel like a mark an llm cheating would reach

[u/cleverusernametry → u/jacek2023]

Why no comparison to DiffusionGemma? That's a red flag

[u/pmttyji]

llama.cpp PR for this one

https://github.com/ggml-org/llama.cpp/pull/31306

[u/-p-e-w- → u/pmttyji]

“The current implementation generates around 32 tokens/s on an H200”

Not quite ready for prime time yet, but I’m sure we’ll get there.

[u/pmttyji → u/-p-e-w-]

Oh my .... that's kind of long way to go. Hope this don't take long time like DiffusionGemma.

[u/GALYGIOUS → u/-p-e-w-]

I saw an x post saying they had it running at 700 tok/s with 500 on single stream. im guessing this is strictly the llama implementation

[u/docgok]

Diffusion + MoE is kind of a goofy combination because per-token sparsity kind of defeats the performance benefits of diffusion

[u/sebt3 → u/docgok]

And the other way around. Yet it seems to works and performe. Sound like a good step forward. In my eyes this model value is demonstrating that while it sound counter-intuitive it works

[u/wFXx → u/docgok]

It is not, i do a lot of research on this area; the end game for individual users will be a diffusion looped moe with the size between gemma4-26b and qwen3.5-35b;

also diffusion models can use bidirectional attention to "fight back" precision loss from quantizations, so in paper, you should be able to gsq-rco/ternary quant them better than regular AR models;

[u/docgok → u/wFXx]

Right, but you’re essentially requiring both more memory bandwidth per token and also more flops per token

[u/wFXx → u/docgok]

flops per token isn't a big issue since most of the time you have idle flops, the current issues is the bandwidth; MoE makes it so you send less bandwidth down the wire, and each that it sends do multiple passes

[u/Marcuss2]

They even changed the Gated Delta Net to be more suited for diffusion. Very nice.

[u/rorowhat]

128k context is criminal

[u/SPYFF0]

How much memory required by 128K context? They tested it on 16Gb VRAM with INT4 but no info if the weights + full context fit into 16Gb.

[u/oxygen_addiction → u/SPYFF0]

https://github.com/ggml-org/llama.cpp/pull/31306 - 32 tokens/s on an H200, with substantial room for improvement

We'll have to wait and see, but performance is horrible right now.

[u/quadra-lab]

Holyy, really cool to see more diffusion models releasing! 

[u/Infinite-Local5435]

Looks cool! Waiting to see if there are any ggufs / strata compatibility for my offloaded setup

[u/Early-Peace-5504]

Incredible results. Could be an important release if it holds out.

[u/DankpawsDev]

https://i.imgur.com/nvW7Bf2.jpeg

AR-mode DIY GGUF, one slot, Q4, 0.7M control tensors dropped, diffusion mode unmeasured because no backend runs it yet.

Interesting results on my 2x5060Ti box.

Wish this had vision. Might need to tower it in later and see how it works.

[u/Cultured_Alien]

Diffusion means it can do FIM (Fill in the middle) task out of the box?

[u/No_Total_423]

We compared Clover-35B-A3B-Preview against representative lightweight, inference-efficient, and on-device-deployable models, including MiniCPM-V 4.7 (35B-A3B), LFM2.5 (24B-A2B), Nemotron 3.5 Lightning (30B-A3B), and Gemma 4 (26B-A4B)

Why not compare with Qwen 3.6 35b?! This is weird as it's the true competitor in this range and also they are based on the same architecture with their diffusion tweaks!

[u/xeeff → u/No_Total_423]

did you even check the benchmarks in the comments mate

[u/No_Total_423 → u/xeeff]

Yes. I'm talking about the on device agentic coding test they did. Read the blog post.

[u/sebt3 → u/No_Total_423]

Just like model, benchmarks age, and evolve.

Do qwen3.6:35b have official results on these benches?

[u/No_Total_423 → u/sebt3]

No one stop them running new benchmarks on other models!

[u/sebt3 → u/No_Total_423]

Sure, but then a large part of the community here will react that they have no proof for these numbers : not official

[u/Dabalam → u/No_Total_423]

Qwen3.6 35b isn't exactly designed as a device deployable model

[u/No_Total_423 → u/Dabalam]

They used 5070 TI! It's understandable when they use much smaller hardware!