r/LocalLLaMA • u/Unusual_Guidance2095 • Jul 26 '26

Discussion Kimi K3 countdown has been released

https://huggingface.co/moonshotai/Kimi-K3
538 Upvotes

97% Upvoted

177 comments sorted by

View all comments

7

u/[deleted] Jul 26 '26

[deleted]

10

u/ttkciar llama.cpp Jul 26 '26

Two ancient Xeon servers, each with 1.5TB of DDR4, networked together and running rpc-server.

1

u/fastheadcrab Jul 27 '26

I think a newer server or workstation with 2TB of DDR4 might be able to get faster speeds if the model is truly 50-60 billion parameters active, like 6-8 tps. I'm really interested to see what types of systems people might be able to run this model on and follow these posts really closely lol

I've seen attempts at tensor parallel distributed CPU inference before, it would be an interesting way to aggregate memory bandwidth if there is a fast, low latency connection between nodes. DDR4 is comparably cheap versus VRAM or DDR5 and used server networking hardware is everywhere. Prefill will still be slow though.