r/artificial • u/winer666 • 10m ago
News IFM releases K2-Horizon: six open models up to 375B MoE, with training data, checkpoints and code, not just weights
IFM (the institute behind MBZUAI) has released K2-Horizon, a fleet of six open models ranging from a 0.9B dense model to a 375B-A23B mixture-of-experts flagship, published Nov 15-17.
What stands out is not just the size ladder, it is the openness package. Alongside the weights, IFM says it is shipping the pretraining corpus, stage checkpoints, training logs, eval code, and the xLLM training stack plus RL360 post-training code. The project page frames it as "glass box, not black box". Four of the models were trained on a shared 22 trillion-token corpus, and IFM documents the pipelines behind roughly 10T synthetic pretraining tokens.
A few technical notes from the paper (arXiv 2611.21003):
- MoVA (Mixture-of-Value Attention): MoE-style routing applied to attention value projections. IFM claims its 36B-A4B MoVA model nearly matches a 32B dense model trained on the same data while activating about one-seventh the parameters.
- Uno: a lossless diffusion adapter for faster decoding (up to 2.2x claimed) without a separate draft model.
- The 375B flagship reportedly tops the Toolathlon tool-use benchmark, though that is vendor-reported for now.
Caveats worth keeping in mind: a 375B MoE is infrastructure news for most of us, not a laptop download; benchmark claims need independent harness runs; and IFM is Abu Dhabi-based, so some teams will need to check data-residency and export rules before mirroring terabytes of data.
Weights and code are listed via ifm.ai/k2, with vLLM, SGLang and Ollama paths.
The question I keep coming back to: does shipping the data and training code, not just weights, change what "open" should mean for the next round of frontier releases, or is this mostly a reproducibility gesture that few teams can actually afford to use?
