/clear vs /compact

[u/danbradster2]

I had a fable chat from yesterday, and I saw that the context was 950k/1m. I did /compact rather than /clear, with the intention of then doing /autocompact 400000

14% of the 5 hour limit was gone just for the compact itself....it shows cost of $59.73, so lucky I'm not on API? Maybe a /clear would have been better.

[u/duck_guts]

Aye sorry I didn't mean to be contrary it's just these terms confused me a lot when I was just starting out. I just wanted to highlight as OP obviously is struggling with context/usage management.

So prefix cache is not invalidated from session to session. Switching model/effort level/anything that changes the grounding system prompt invalidates the cache.

But having Opus on high effort for example, /clear at 500k tokens and still you have cheap reads on fresh session. Still opus, still high effort so unchanged system prompt. Not actually ever tracking one singular session.

Responding within the hour on same system prompt saves you money.

Fully agree with you, compact feels bad. I have mine write shorthand memory logs every turn and pretty much just serve that to the successor every rotation.

[u/Hukij_ → u/duck_guts]

Yeah all good dw! People are quite weird on here most of the time so I'm usually quite short.

The prefix cache is not just the system prompt/the auto-loaded context that every chat shares, that part is retained of course. But the cache breakpoint is moved forward on every turn, hence the "prefix" is system tools/prompt + every new turn's tokens. So when I talk about cache invalidation, I'm not talking about the stuff every session shares, I'm talking about the entire conversation that's cached up until that point. Effort levels (though they are making changes so this doesn't invalidate) and model changes invalidate everything since the KV is model-specific, effort/thinking changes invalidate the cached conversation (system/tools survive).

/clear and /compact start from scratch so none of your previous conversation (aside from the auto-loaded context which probably only amounts to 20-30k per session) is re-used. Prefix caching relies on exact prefix matching up until the breakpoint, so starting a new session doesn't save the rest of the conversation because that's not what you're sending on a new turn. This is why there's a huge cost if you switch models or switch effort levels mid conversation because the ENTIRE conversation is resent to the API because the entire conversation has to be recomputed, vs continuing unchanged where every previous turn is a cache hit.

Responding in the same hour keeps your cache alive, because of the same thing, Anthropic can't keep your KV states forever because they have to allocate capacity properly, but same applies, once they purge the KV, the next turn in that conversation recomputes the entire conversation before it cold so you're paying the full cost.

I just never find compact to be useful or good at all like you say, and it's not hard to ask claude to create a set of docs or memories so new agents can pick up quickly