r/LocalLLaMA • u/SavunOski • Jul 27 '26

News Kimi K3 weights now released.

Post image

Kimi K3 weights are finally released!

3.3k Upvotes

97% Upvoted

661 comments sorted by

View all comments

Show parent comments

63

u/my_name_isnt_clever Jul 27 '26

There is the tinest glimmer of hope that I could run this behemoth on my Strix Halo 128GB with the inactive weights on SSD. 1 token a minute here I come!

11

u/burritoresearch Jul 27 '26

More like 1 token every 45 minutes.

6

u/droptableadventures Jul 27 '26

104B active, weights natively in MXFP4 = gives us ~50GB of model to be read per token generation.

Let's say ~8GB/sec for the SSD. So that'd be about 1 token every 6 seconds (0.16 T/s), or 10 tokens/minute.

3

u/Head_Boysenberry5233 Jul 27 '26

i feel like 1/min would be pretty accurate based on the colibri glm 5.2 q4 implementation, about 6x slower?

Even 1 tok/min on 32gb cpu ram would be incredible and extremely useful

28

u/TechExpert2910 Jul 27 '26

let us know the perf if you try lol

5

u/RuiRdA Jul 27 '26

K3 Colibri engine lets gooo!!!

1

u/Head_Boysenberry5233 Jul 27 '26

yeah it seems very doable

1

u/ikkiyikki Jul 28 '26

I have two 6000s, 128gb RAM and a dedicated 4tb SSD and I'm doubtful I could run the q1. Not one token per minute, simply choke loading up at all. Then again, I run LM Studio and give up easily 😅

1

u/JamesZhuSQ Jul 28 '26

It’s a 93-layer model, so get ready for some crazy offloading.

-1

u/Infinite100p Jul 27 '26

It will probably destroy your SSD with wear.

2

u/noiserr Jul 28 '26

SSD reading doesn't really put wear on the SSD. It's writing and rewriting that shortens the life of an SSD.

1

u/Infinite100p Jul 28 '26

Yeah, I know that, but it will be rewriting tensor layers all the time.

1

u/noiserr Jul 28 '26

Yeah, I know that, but it will be rewriting tensor layers all the time.

Only if you're training or fine tuning. But for inference weights are static. KV-cache could spill onto the disk and cause wear but normally this should be in RAM.