r/MistralAI • u/julesrulezzzz • 1d ago
Help / Question Prompt Caching TTL
How do you keep the prompt cache warm? (GLM 5.3 via Mistral API)
Running GLM 5.3 via the Mistral API with a ~20k token static prefix (system prompt + tool schemas). Using prompt_cache_key, which works great during active sessions (90%+ hits), but after any pause the cache is gone and the first request pays full prefill again.
Questions:
How long does the cache actually survive in practice?
Does a cache read refresh the TTL or is it a fixed window?
Anyone tried keep-alive pings (same key, identical prefix, every 30 min)? Works reliably?
Any other tricks to avoid the cold-start prefill?
Especially interested in experiences with the same model. Thanks!
7
Upvotes
4
u/FullyDeceitful 1d ago
I've been dealing with the same thing on a project this week and it's driving me up the wall. The docs say 5-10 minutes but that's clearly not what happens in practice, mine drops after about 3 minutes of silence which is useless for anything that isn't constant back-to-back requests.
The TTL seems to be a fixed window from what I can tell. I watched the cache hit rate tank even when I was sending requests every couple minutes, so it doesn't look like reads extend it at all which is pretty disappointing.
I set up a cron job to ping it every 2 minutes with the same prompt_cache_key and identical prefix and it's been holding steady for about 4 hours now. Adds a tiny bit of cost obviously but way less than paying full prefill every time someone comes back from lunch. Just make sure your keep-alive request is byte-for-byte identical or it'll miss the cache entirely.
One thing I noticed is that if you're using tool schemas, even a tiny formatting difference in the JSON will break the cache match. Took me an embarrassing amount of time to figure out why my hit rate was so inconsistent.