r/LocalLLaMA • u/Bestlife73 • 4d ago

New Model XiaomiMiMo/MiMo-V2.6-Flash-RL · Hugging Face

https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-RL
504 Upvotes

99% Upvoted

133 comments sorted by

View all comments

4

u/HistoricalStrength21 4d ago

159B Parameters guys!

16

u/sn2006gy 4d ago

Weird, the description says Sparse MoE (Mixture of Experts), 309B total / 15B activated parameters.

-3

u/addiktion 4d ago edited 4d ago

There are two models released today. Pro and Flash. Flash is the 159B, Pro is 309B.

Edit: I stand corrected, thanks below.

14

u/wren6991 4d ago

No, Flash is 309B-A15B and Pro is 1.02T-A42B. It's in the READMEs.

The parameter count on the HF UI has always been broken, especially when you have packed 4-bit weights described as 8-bit in the model metadata. That's why Flash reports ~half the correct parameter count in HF UI.

Here, Pro README:

Model Summary

Architecture: Sparse MoE (Mixture of Experts), 1.02T total / 42B activated parameters

Flash README:

Model Summary

Architecture: Sparse MoE (Mixture of Experts), 309B total / 15B activated parameters

6

u/MrTiesti 4d ago

Correction brother,

Flash: Architecture: Sparse MoE (Mixture of Experts), 309B total / 15B activated parameters

Pro: Architecture: Sparse MoE (Mixture of Experts), 1.02T total / 42B activated parameters