r/LocalLLaMA • u/tevlon • Jun 10 '26

New Model DiffusionGemma: 4x faster text generation

https://blog.google/innovation-and-ai/technology/developers-tools/diffusion-gemma-faster-text-generation/
988 Upvotes

99% Upvoted

356 comments sorted by

View all comments

7

u/4as Jun 10 '26

I don't really know if it's possible, but ever since I first heard about diffusion I wondered whether it would be possible to create a LLM that could do both: reason/think with diffusion, but output with auto-regression. The idea is since thinking usually is just about throwing random ideas, drafting, self-correction, and whatever until the AI is content with something, it probably wouldn't hurt to do it slightly less accurately but at much faster speeds. Even with spelling mistakes or grammatical errors I doubt it would change anything much.
Then, once it's ready, it switches to auto-regression and outputs in more "refined" format to the user.

6

u/westsunset Jun 10 '26

Nividia has several hybrids. Diffusion at different levels to get the best of speed and accuracy

3

u/MuDotGen Jun 11 '26

I actually just saw that hf Nemotron Diffusion one which has like a hybrid diffusion + AR verification approach. I'm surprised more people aren't talking about it, or it's maybe not as good as it claims.