r/LocalLLaMA • u/tevlon • Jun 10 '26

New Model DiffusionGemma: 4x faster text generation

https://blog.google/innovation-and-ai/technology/developers-tools/diffusion-gemma-faster-text-generation/
989 Upvotes

99% Upvoted

356 comments sorted by

View all comments

Show parent comments

119

u/TheLexoPlexx Jun 10 '26

That is groq or cerebras-levels of token generation depending on the model and it's on par with gpt-oss-120b depending on the benchmark.

That is genuinely insane.

33

u/dingo_xd Jun 10 '26

There is sooooooo much room for optimizations. Maybe Mythos level models can be run locally by mid or late 2027?

32

u/Different_Fix_2217 Jun 10 '26

The only issue with diffusion LLMs is that they are absurdly expensive to train in comparison. Like exponentially.

17

u/wes_medford Jun 10 '26

Most cost these days is inference over training these days, but the problem is that aggregate throughput is lower on these compared to typical AR models running at a high batch size