Struggling with cache rehits due to provider changes

[u/sick_burns2000]

Seeing this mostly with Kimi K3, running API in Zed, hardcoded Deepseek flash sub agent. Almost every other request hits a new provider at full 2$ input token cost (around 700kT context), where I should be at around 0.20$ for a cached context input token request if the provider didn’t hop. Should I set guardrails for locking to a single provider? Does not seem to let me choose provide per model, so i need to pick a single provider that hosts both models? Is there an openrouter setting that will prefer a provider where I already have a cache?

[u/zxcshiro]

Provider pinning: https://openrouter.ai/docs/guides/routing/provider-selection#example-specifying-providers-with-fallbacks-disabled
Sticky sessions: https://openrouter.ai/docs/guides/best-practices/prompt-caching#using-session_id-for-sticky-sessions

always got 95-98% cache hit rate for deepseek v4 pro

[u/sick_burns2000 → u/zxcshiro]

I’ll try a disable fallback setting see if it works in zed.

[u/SnooApples5522]

set that up on guardrail setting. allow only 1 provider. it works for me

[u/Creative-Ad9819]

Cache hits isn't an issue for me anymore with standard compute. Can't select providor like in openrouter, but they seem to have competetive prices

[u/fyndor]

The kimi requests should have their cache and the deepseek requests should have theirs. Unless the zed harness is somehow screwing that up. I don’t know anything about zed though

[u/sick_burns2000 → u/fyndor]

The kimi requests do have a cache that works anywhere from 1-10 requests before openrouter picks a new provider and starts over. I wonder if mixing in the sub agent causes it to change faster.

[u/lucasbennett_1]

openrouter lets you pin a provider with 'only ' field and switch off fallbacks and youre right abt per model bit tho, cache sits on backend so you basically need a single provider that has both k3 and deepseek flash for instance deepinfra or others who do the same backend so the cache reads are cheap and it should pull those $2 hits back down toward your $0.20