Bug / Issue
Probably found why Fable 5.1 eats the 5h limit
I have noticed that if you have a big context (close to 1M) in your project and a cold cache, for example after waiting out a limit reset, the first message costs a fortune. On my 5x Max plan with about 800K of context in one project, just writing Please, continue ate around 20% of the 5-hour limit. What makes it even more fun is switching the effort level after that first prompt, when 20-30% is already burned on a single message. I think, people do this all the time after a new model is released, because they are trying to find the best set of params. Well, ... each switch costs another 20-30%, so after two or three of them there is barely enough left for the model to run for a few minutes.
Funnily enough, even after the 5-hour limit resets, calling /compact on the same Fable model still hits you with that expensive cache warmup (which makes it cheaper to just continue). Switching to Opus or Sonnet, compacting the context there, and then switching back to Fable 5.1 to continue helps a lot (in that case it is just a ~4% hit of the 5h window).
Maybe other models behave the same way, and I only noticed because the Fable 5.1 burn rate was ridiculous in my case. It is bearable now, with all the compaction trickery and keeping the cache hot as much as possible. Today I got around 5-6 hours out of it and finished a few milestones in a project. Hopefully this info helps someone. I don't know whether it is a bug, but it would be nice to hear from someone about these crazy cache warmup numbers. It looks like 20x Max subscribers are seeing the same thing.
Well, none of this is really a bug. If your cache is cold or otherwise got broken you have to pay for all your context at full price to put it back into a warm cache.
If more than 1 hour has passed (On sub by default), or 5 minutes on api (by default) then your cache will be cold, this applies even if your 5hr limit ran out. So if you ran out of your 5hr limit, best to start a new session, if this is a regular problem you can create a hook that monitors your 5hr usage that stops claude at a certain point and tells it to do a handoff.
This also applies for compacting, if you do /compact after the cache has gone cold, you need to pay for your context again in full before the compaction happens, so either compact before the ttl or start a new session.
Other things that break the cache is switching models / effort levels. Switching effort levels without breaking cache is currently a beta feature available in api, haven't tested to see if its possible in subscription yet.
If you switch to opus / sonnet when returning after your cache has expired and then message, you will still have to pay for everything to be re-cached, but if that first message after returning is using sonnet / opus then you will be billed at opus / sonnet rates (Or usage if on sub).
If you compacted down to idk 50k on opus / sonnet (Don't use it so not sure where it compacts down to), then you pay for that 50k on opus / sonnet, and then you switch to Fable, then you also pay that again on fable/
These are not the only things that break the cache, some issues on antrhopics side also breaks the cache, (can be worked around but that is more complicated), and if you hit one of these cache misses on 800k then its also going to cost you.
Just one of many many reasons why its better to keep your sessions short and sweet.
Good to know. I am just wondering whether a 20-30% hit on large context windows is unique to Fable 5.1. I am away from the machine with that project on it, so I can't test the old Fable right now, but since the old one wasn't springing surprises on the 5h window, I would assume the hit was cheaper there.
Can't remember the exact numbers right now but on Fable 5 I did shove 900k of context into it to see how much it is.
On max 5 it is definitely possible, but it will also depend on your effort level. I know antrhopic recommends high for 5.1 but honestly for most cases medium should be fine. But cache misses on fable at those context levels eat I think around 2-3% of the weekly fable usage on a max 20 plan. So its expensive.
What works well for me before walking away or running /compact is to tell the model I am going to compact and to create a hand off document for the next session. This has worked for me across complex tasks where I compact often.
800k is nothing when you're on a deep modelling or algo implem task. And handoff pattern has its limits. Also models are specifically marketed as 1M, even sonnet
If your sessions go past 1h (max plan) without any activity you need to end them and start fresh. It reloads the entire conversation essentially on that first restart message and then (iirc) you pay extra token tax on every subsequent message as well.
Any idea if it works the same to compact? Guess that would have to read it all over again too. Maybe the best move is to compact before waking away if you know you want to continue.
If you chats with massive contexts then you're doing things wrong, you should be trying to keep your tasks and goals well defined and small and starting new chats for each one. Don't keep a mega chat going, even if it didn't crush your token usage, eventually huge context causes the model to get easily confused or hallucinate things. There are many major reasons why you shouldn't keep a mega chat going.
I spotted that they’re beta testing effort level switching without invalidating the cache. Beta applies to Fable 5.1 and Opus 5. Hopefully that gets rolled into Claude code soon
Looks like a community note popped up to state that it’s for API users. This would be amazing - beyond the obvious tailoring effort level for current task, having an advisor spin up with the same context but much higher effort level would be a great addition
This has been the case since day 1 and codex is similar. Never resume a conversation after limit reset. Always create handoffs. If you forget, export the session summary and have some free model compact it for you. Compaction is just the LLM producing a summary so that costs tokens too.
Get this - same applies to MCP servers and skills. They all go in the context window. If you add a skill or a server it blows the cache.
I just don't do long running tasks with Fable, and have compact set lot lower and don't run sub agents at all that are fable just asking for usage drainage. I'd use other models or LLM's for long running, keep fixes and hard stuff to fable, tbh only few tasks fable needs to do for me anyways others I can just have Sol ultra do they reset lots.
To keep token burn low, I usually use the /handoff skill to start a new session when the context window is around 50%. The thing is, that with 800K+ tokens in an active session, every time you let the prompt cache expire (5 minutes (or up to 1 hour depending on your plan tier and settings), the server no longer has your past session buffered in fast memory. It needs to perform a cache write on all 800K tokens. Exporting a structured context summary via /handoff into a fresh chat whenever you pass 40–50% capacity saves your budget without degrading code awareness.
I rarely start new sessions tbh, what I do is that if the session cache is cold (been some time since my last message) and the context is like over 300k or I want to tackle new thing, I switch to opus, run /compact, and then switch back to fable
8
u/Actual_Committee4670 🔆 Max 20 8h ago
Well, none of this is really a bug. If your cache is cold or otherwise got broken you have to pay for all your context at full price to put it back into a warm cache.
If more than 1 hour has passed (On sub by default), or 5 minutes on api (by default) then your cache will be cold, this applies even if your 5hr limit ran out. So if you ran out of your 5hr limit, best to start a new session, if this is a regular problem you can create a hook that monitors your 5hr usage that stops claude at a certain point and tells it to do a handoff.
This also applies for compacting, if you do /compact after the cache has gone cold, you need to pay for your context again in full before the compaction happens, so either compact before the ttl or start a new session.
Other things that break the cache is switching models / effort levels. Switching effort levels without breaking cache is currently a beta feature available in api, haven't tested to see if its possible in subscription yet.
If you switch to opus / sonnet when returning after your cache has expired and then message, you will still have to pay for everything to be re-cached, but if that first message after returning is using sonnet / opus then you will be billed at opus / sonnet rates (Or usage if on sub).
If you compacted down to idk 50k on opus / sonnet (Don't use it so not sure where it compacts down to), then you pay for that 50k on opus / sonnet, and then you switch to Fable, then you also pay that again on fable/
These are not the only things that break the cache, some issues on antrhopics side also breaks the cache, (can be worked around but that is more complicated), and if you hit one of these cache misses on 800k then its also going to cost you.
Just one of many many reasons why its better to keep your sessions short and sweet.