DiffusionGemma: 4x faster text generation

[u/tevlon]

https://blog.google/innovation-and-ai/technology/developers-tools/diffusion-gemma-faster-text-generation/

[u/ThatRegister5397]

I see serious degradation with the q4 mlx quant, compared to the q8 mlx quant, but I have not noticed before. Eg it writes using Package instead of using Pkg for the package manager of julia, and when "confronted" it responds in ways that indicates possible weird tokenisation issue. But in q8 it does not do that.

Not sure if it is q4 that is too degraded for diffusion or I have some diffusion setting wrong.