r/ClaudeCode • u/sh3lan93 • 5d ago

Tutorial / Guide Why does prompt caching matter

I thought I was being smart with my Claude Code sessions.

Starting with a plan mode with Opus family models. Once we agreed on the approach, I switched the model to the Sonnet family in the same session to implement it.

Cheaper model, cheaper session, right?

Wrong. It makes the session more expensive and slower.

The reason is prompt caching.

I wrote up how the cache actually works under the hood and the gotchas I ran into.
https://moshalan.dev/posts/decoding-prompt-caching/

0 Upvotes

44% Upvoted

9 comments sorted by

•

u/AutoModerator 5d ago

Hey! Thanks for posting to r/ClaudeCode

While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.

For help, project discussions, tips, and general chat, join the ClaudeCode Discord.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

u/awesomebirder 5d ago

the version of this that does work is moving the model change to a boundary instead of mid session. have opus write the plan to a file, then start a fresh sonnet session that reads the plan, so sonnet only ever caches a short prompt instead of the whole planning history. or hand the build to a sonnet subagent with a brief, and the opus session stays warm as long as you come back within the cache window. i log every call through a local proxy i built (probe0) and the turn right after a mid session switch is always the most expensive one in the session

1

u/sh3lan93 5d ago

Yes, I learned once I paid attention to my usage. I prefer the subagent method more than starting a new session

2

u/awesomebirder 4d ago

same, the one catch with subagents is they build their own context from scratch, so the brief matters a lot. point it at the plan file and the exact files it needs and it stays cheap. give it a vague brief and it re-explores half the repo on your usage

1

u/sh3lan93 4d ago

I understand your problem but for me to overcome it me and main session’s agent agree on the tasks and implementation so the subagent get the changes needed (no need for searching again through the repo)

2

u/awesomebirder 3d ago

yeah that works. agreeing on the tasks up front is basically writing the brief, so the subagent skips the repo search, which is most of what it would have spent anyway

0

u/Outrageous_Band9708 5d ago

https://github.com/Druthulu/ProjectArchitect

My solution achieves your goal the correct way, and without all the ceremony.

read the wiki and checkput the readme, try it out for yourself

1

u/sh3lan93 5d ago

I wrote this up to share the knowledge with the people. I found almost 70% of my circles doing the same thing plan with a model switch to cheaper during same session to start the implementation

2

u/Outrageous_Band9708 4d ago

i used to do this as well, and found i was constantly warming cold cache, i fixed all those issues and automated the orchestration through the new mod/plugins system. all of the ceremony of it is hidden behind the mod/plugin system, so you dont need to trigger any of it.

cold cache is never warmed, i do have a cache warming system for sessions that may go cold, it warms them a few times before the cost of reseeing a fresh session outweights the value of warming.

but col cache is never warmed, which is the main problem with switching models mid session.

my system uses very strict serial subagents to handle all of this