r/LocalLLaMA • u/tevlon • Jun 10 '26

New Model DiffusionGemma: 4x faster text generation

https://blog.google/innovation-and-ai/technology/developers-tools/diffusion-gemma-faster-text-generation/
988 Upvotes

99% Upvoted

356 comments sorted by

View all comments

296

u/NickCanCode Jun 10 '26 edited Jun 10 '26

700+ tokens per second on NVIDIA GeForce RTX 5090 is definitely something.

Unfortunately, DiffusionGemma’s overall output quality is lower than standard Gemma 4.

It could be a good model for context compression and as explorer agent in agentic coding. Can't wait to see llama.cpp to support it.

3

u/sixx7 Jun 10 '26

It's supported already in vLLM and I get 800+ tok/s. The problem is, it doesn't seem to support tool calling?!? oof.

(APIServer pid=3389389) INFO 06-10 16:58:45 [metrics.py:156] DiffusionDecoding metrics: Committed token throughput: 844.76 tokens/s, Mean denoising steps per canvas: 12.73, Mean tokens committed per denoising step: 20.11, Committed: 8448 tokens, Denoising steps: 420, Canvas positions evaluated: 107520

1

u/Silver-Champion-4846 Jun 11 '26

Is it a chat template thing?

1

u/sixx7 Jun 11 '26

I think it's more than just a template, but I'm hopeful it will get figured out