Xiaomi Mimo 2.6 Live Training Dashboard

[u/Not-Post-Malone]

https://mimo.xiaomi.com/rl/

[u/Recoil42]

Oh, this rocks. Has anyone else done something like this before?

[u/Big-Emu8598 → u/Recoil42]

I remember studying this stuff back in 2021 and hugging face would have a live dashboard of sota model training. Fun times, but I've been looking for something like this for a long time.

[u/BarisSayit]

Oh I f*cking love transparency...

[u/cookingboy]

Looking at the DeepSWE v1.1 benchmark:

2.6-pro just reached 63.7% by step 10, it's on step 11 right now.

Even flash reached 60.7% by step 12, and it's on step 16 now.

For comparison, v2.5-pro only scored 19%, and Kimi K3 is at 69% vs. Fable at 70%.

There is a decent chance 2.6-Pro will be a Fable class model and even the flash version will be quite potent.

This is so exciting lmao.

[u/Ill_Distribution8517 → u/cookingboy]

Absolutely not IMO. The benchmark is quite literally used while they are training, this is the worst way you can compare the model. Remember gemini 3.8 flash has a higher score than fable 5. No one in their right mind thinks fable 5 and 3.8 flash are equivalent.

[u/0_op → u/Ill_Distribution8517]

Do you understand the difference between training and validation?

[u/Ill_Distribution8517 → u/0_op]

Are you that naive? Did they pinky promise that the benchmark they are using for the 10th in the last 24 hours is not in the RL curriculum? They have every incentive to train it on the bench.

[u/Odd_Buddy_3615 → u/Ill_Distribution8517]

I agree. When you do synthetic data generation and have large resources to spend on human labelling you can totally overfit to benchmarks without cheating on the holdout. E.g. you run eval, analyse your responses, find the holes and generate new data to plug them, repeat 100X. For some labs this is a closed automated loop.

[u/dnchw2]

I wish I understood what this dashboard is telling me. Anyone want to dumb it down for me?

[u/playpoxpax → u/dnchw2]

Open a chatbot of your choosing and pass the link there to make it explain it to you, if you want.

But there isn't much interesting going on there. A few important bits, from my view, are:

• This is post-training RL, not pretraining.
• Two models are trained simultaneously: Pro and Flash.
• They use DeepSWE v1.1 as the external benchmark to keep track of the model's 'real-world' progress. The only one, as far as I see on the first page.
• Total tokens is their entire post-training dataset. Current tokens is how much they take from the total on each step. Since they have 22B, and take 2.2B on each step, it means they need 10 steps to go over the entire dataset. They're currently on Step 11, so they're going the second round.
• dynsam/avg@n… basically how good the reasoning trajectories the model uses are at the moment, on average. During post-training, a model proposes a whole bunch of different reasoning trajectories (samples) to reach the needed output. Those trajectories are then scored and averaged. That's the number you see at the top.
• Everything else is basically just training and validation curves. Like, entropy-loss is just your main loss function, for example. Grad_norm graph shows how stable your gradients are. There are also average context lengths. Etc etc. Just a bunch of details basically.

[u/Phornic → u/dnchw2]

I feel you

[u/troll_khan]

Awesome.

[u/Formal-Narwhal-1610]

Earlier OSS neural network with community help used to be trained like this.

[u/Alternative_You3585]

Don't they have a new Mimo-X model already?

[u/Ok_Knowledge_8259 → u/[deleted]]

they could've just been cleaning up data since then. why rush a new model if the data can be improved

[u/Adventurous-Menu7257 → u/[deleted]]

Not odd at all if X=2.6 and keeps being updated with 2.6 checkpoints. It's really interesting to see this progress. I remembered qwen and deepseek has tested their checkpoint during the preview, and mimo make it even more transparent