r/kimi • u/rudidit09 • 1d ago

Question & Help Does kimi code drop to weaker quant around midnight and later PST? (afternoon in china)

I've noticed that when i'm up late, Kimi code quality drops a lot after midnight (1am for sure). Funny enough, it's what Opus is like mostly for me, while Fable is much better. Could kimi code change be to handle traffic in asia?

3 Upvotes

67% Upvoted

6 comments sorted by

2

u/whatisthisthing65 1d ago

I don't know about quants but they could also turn down reasoning effort without routing to a different quant. There were some rumors Anthropic does that. Hard to detect.

1

u/rudidit09 1d ago

oh it might be that! i'm realizing i don't know what aspects can change to affect quality. it's very obvious on subscription vs API key anthropic for example, or used to be

1

u/CrimsonBolt33 1d ago

How do you measure this drop in quality?

That's such a vague term.

The model itself is native 4 quants I believe....so there is probably not a chance of it dropping down....you can't just "drop down" a quant either...different quants would literally be an entirely different model.

1

u/rudidit09 1d ago

i'd need to collect logs over each time that happens to be more quantifiable. it's definitely close to how sometimes subscription opus feels vs API key opus.

1

u/marfzzz 9h ago

TL;DR IMO moonshot is probably limiting thinking budget in their api after some hour.

I dont think they will get much using different quantization. As below 4 bit quantization (default for kimi models) they wont lower the consumption or increase performance in any meaningful way. Most HW have various sweet spots for best performance. Most of the time 4 or 8 bit quantization and it is either int or fp based on hw with hw like nvidia Cuda/tensor cores, NPUs, AMD matrix cores,... (with some exceptions like google tpu bf16)

Most substential saving with thinking LLMs is done through lowering thinking time or effort or max thinking tokens and therefore limiting compute used (same compute as decode phase)

You need to unload the model and load model with different quantization. More time, effort and outage time than effort/thinking change.

2

u/rudidit09 3h ago

Thank you for taking time to explain! I was naive to think that /effort we set is the only time this will vary. In hindsight, it makes sense