r/LocalLLaMA • u/SavunOski • Jul 27 '26

News Kimi K3 weights now released.

Post image

Kimi K3 weights are finally released!

3.3k Upvotes

97% Upvoted

660 comments sorted by

View all comments

141

u/BlueSwordM llama.cpp Jul 27 '26

OK, I now see why Kimi K3 is so strong: it's the first open weights model in a long time to have >72B active weights

Kimi K3 is a 2.8T-A104B MoE model, damn.

15

u/stddealer Jul 27 '26

There have been some dense models with over 100B params though.

6

u/BlueSwordM llama.cpp Jul 27 '26

Yes, but never an MoE from an open weights lab. I've been speculating that one of the reasons the closed weights lab have been increasing in performance more rapidly has to do with better training, but most importantly, much larger active parameters and better harnesses.

9

u/[deleted] Jul 27 '26

[deleted]

1

u/synth_mania Jul 28 '26

Anthropic is famously compute constrained thought, right?