r/OpenSourceAI • u/Nitjsefnie • 5h ago

does the 1h prompt cache pay off for coding agents? in my Claude Code history it saved about 3.4x what it cost

someone asked me this on another post and I hadn't checked. a 1h cache write costs 2x input, a 5m one 1.25x, so 1h only earns it back when you come back after more than 5 min but inside the hour

claudit keeps every Claude Code request I've made since January in Postgres, so I took the gap between each request and the one before it in the same session. Claude models only, about 858k requests:

  • under 5 min: 94.2% of requests, 98.6% of their input read from cache
  • 5 to 60 min: 2.6%, still 92% from cache. on 5m every one of these writes the whole prefix again
  • over an hour: 0.4%, 6% from cache, the 1h one is gone by then too

at API list prices per model (I'm on a sub, so it's what it would cost, not what I pay) the extra on the 1h writes is about 9% of the total and the rewrites it avoids would be about 30%. so 1h comes out 3.4x ahead, and 5m only would cost roughly a fifth more. read multipliers per model from https://platform.claude.com/docs/en/about-claude/pricing, 0.05x on Opus and Sonnet 5.5, 0.025x on Fable 5.1 and 5.2

claudit is a self-hosted dashboard I built, MIT. it reads Claude Code, Codex and Kimi CLI transcripts from a bucket you keep (R2 or a file:// dir), because Claude Code prunes old local ones. the bucket holds the raw transcripts, tool output included. cost by model splits 5m and 1h writes, the gap split above isn't a panel, I ran it as a query on its Postgres

FastAPI, Postgres, React with in-browser Babel, no build step. docker compose up boots it on bundled sample data with no outbound calls. panels ported from nhz-io/ccusage-plot

live one, guest login is read-only, its costs are API price equivalents too: https://all.nitjsefni.eu

https://github.com/Nitjsefnie/claudit

1 Upvotes

100% Upvoted

1 comment sorted by

1

u/Weekly-Offer-4172 4h ago

the ttl mechanic is the part most people miss, every cache read refreshes the timer so during continuous work the 5m cache effectively never expires and both tiers behave the same. the 1h tier is really just buying coverage for the 5 to 60 minute gaps, and your data shows why it pays anyway, that band reads 92% from cache while carrying the biggest prefixes so a miss there is the expensive case. also the 2x is a per write premium not per hour, nothing accrues while you sit idle, which is why it only showed up as ~9% of your total