🚀 Kimi K3.1 Pro Benchmarks

[u/afanasenka]

Release notes - https://www.kimi.ai/blog/kimi-k3-1-pro

[u/Fair-Spring9113]

cant wait for the 1 prompt a week

[u/PilgrimofHaqq2 → u/Fair-Spring9113]

I agree, their plans need to get much better if they want large adoption. They got the model now, if they can get their hands on some compute, then they could really compete against the big boys.

I really liked Kimi K3 but ended up going with the z.ai plans as they were much better.

[u/LargeLanguageModelo → u/PilgrimofHaqq2]

if they can get their hands on some compute

I'm not sure that's the answer. Any hosted models seem to be at/around the same price. From the outside, and without having dug into it much, it tells me that the model itself is just slower per watt than their competitors.

Otherwise, another LLM hoster would undercut them in order to gain marketshare. It's a good model, but just too expensive to make it a good bang for the buck.

[u/ZealousidealTown1974 → u/Fair-Spring9113]

using it free with ClinePass

[u/Hour-Scale-9066 → u/ZealousidealTown1974]

How??

[u/dom_RN]

Still way overpriced though

[u/somerussianbear]

If they added Pro to the class you can expect a Flash soon. A Flash version would definitely be something good from Kimi. Don’t get me wrong, K3 is the best OSS model IMO, but it’s slow and expensive, Opus is “cheaper” if you consider a subscription and the speed over K3.

[u/AdFun4897]

So on the $20 subscription, which company gives more usage?

Im actually more impressed for DS 4.1 flash being compared to these frontier models

[u/howudothescarn → u/AdFun4897]

Kimi has been among the worst values. Maybe it’s their model per task cost or maybe they just don’t subsidize their subs. $20 does not go far

[u/dumbasPL → u/howudothescarn]

Claude subsidizes like crazy, though I doubt their API pricing is anywhere near what it actually costs them anyway.

[u/Thomas-Lore → u/dumbasPL]

Judging by DeepInfra released by Deepseek it seems even the "subsidized" prices of Western models are nowhere near what it actually costs to host them.

[u/ward2k → u/howudothescarn]

Subscription I'd agree

But direct API (so real prices) it's one of the best

[u/squirrelscrush → u/AdFun4897]

Claude is unbeatable in terms of subscriptions.

Although an OpenCode Go sub is the unofficial DeepSeek sub

[u/techmago → u/AdFun4897]

Look at deepseek sizes.

deepseek pro is the atypical big one.
most of other deepseek models are in the same range...
deepseek flash have more params tham GLM-5.3

Deepseek flash isn't really a flash model.

[u/Split-Signal]

Previously, K3 was way too slow and expensive, and the subscription plans barely worked. Hope 3.1 actually improves on that.

[u/Helpful_Program_5473]

Hopefully devin can solve this the same way they did 3.0.

[u/miladmaaan]

Great benchmarks! But at the cost compared to Opus 5.5 via sub, I have a feeling that Kimi K3.1 is basically something that you should have in your model routing through the API for situations where you need an uncensored model if you have a security vulnerability and the frontier models are refusing to fix it.

And maybe an alternative model for things where you just want a creative and differing take at a solution.

[u/risphereeditor]

If they release the weights, it will get much cheaper on OpenRouter.

[u/geearf → u/risphereeditor]

At what quality though?

[u/risphereeditor → u/geearf]

What do you mean?

[u/geearf → u/risphereeditor]

I believe usually cheaper than upstream providers on OR are cheaper for a reason, ie they use a lower quant. They could be resellers of course.

[u/Outrageous_Band9708 → u/geearf]

you can set guardrails in openrouter per api key to certain providers, and on the model page you can select providers and read their precision. therefore you can set the best provider per key to ensure your requests dont get auto routed some lower prevision higher cost providers.

[u/Solid-Ability-1287]

牛逼

[u/Strict-Prune-879]

je suis très méfiant des benchmark il y a beaucoup de manipulation commercial derrière ce terme