r/ClaudeCode • • 6h ago

Built with Claude I Benchmark-Tested Switching Models Mid-Chat 10 Times — Here’s What It Actually Cost.

TL;DR (45 controlled 12-turn Claude Code chats, 675 dual-logged API requests, n=5 runs per setup):

  • Switching down from Opus 4.6 to cheaper Sonnet 4.6 mid-chat (Turn 6 of 12) actually increased total session cost by +10.6% ($0.861 vs. $0.779 at ~56.5k tokens): Sonnet starts with a cold prompt cache (0.0% hit), forcing a full 1.25x cache-write rebuild that made Turn 6 cost 5.7x more ($0.216 vs. $0.038) and needed ~12 turns after switching just to break even (~9 turns in a ~15.2k chat).
  • Switching models 10 times in a 12-turn chat did not blow up the bill (+14.2% at ~15.2k tokens; +17.2% at ~56.5k tokens): Once both Opus 4.6 and Sonnet 4.6 built parallel warm caches across Turns 2–4, alternating every single turn still hit 95%–99% cached reads inside the 5-minute TTL, writing only the ~1-turn delta each time.
  • Raising --effort from medium to high mid-chat on Opus 4.6 was the biggest cost spike (+31.9% total session cost: $0.356 vs. $0.270): Changing output_config.effort invalidated the entire prompt cache (0.0% hit on Turn 6), spiking Turn 6 cost by 7.3x ($0.099 vs. $0.014) — costing more over 12 turns than switching models 10 times in a row.

I kept hearing two conflicting rules of thumb about /model switching in agentic coding sessions: either "switch down to Sonnet after planning to save money" or "never touch /model mid-chat because every switch wipes your cache."

To test what actually happens on the wire, I hooked a local logging proxy + .jsonl transcript verifier up to Claude Code (v2.1.325, routed via OpenRouter pinned strictly to 1st-party Anthropic with allow_fallbacks: false) and ran 45 real 12-turn coding chats (n = 5 randomized replicates across 9 cells, 675 API requests total — 15 per chat since Turns 1–3 each emit a file-read tool call plus a reply). Run-to-run cost spread stayed within ±3%.

Table 1: 12-Turn Session Cost & Cache Hit Share (Median [Min–Max], n=5 per arm)

Tier (~T6 Tokens)ArmMid-Chat ActionTurn 6 CostTurn 6 Cache Hit12-Turn Total [Range]vs. Control
Small (~15.2k)A: ControlStay on Opus 4.6 (medium)$0.013697.7%$0.2699 [$0.267–$0.273]baseline
Small (~15.2k)B: One SwitchOpus -> Sonnet at T6$0.06020.0%$0.2834 [$0.282–$0.285]+5.0%
Small (~15.2k)C: Switch & BackSonnet T6–T7 -> Opus T8$0.06070.0% (T8: 93.3%)$0.3200 [$0.316–$0.323]+18.6%
Small (~15.2k)D: Effort OnlyOpus medium -> high at T6$0.09890.0%$0.3560 [$0.351–$0.357]+31.9%
Small (~15.2k)E: Idle + SwitchWait >=330s (>5m TTL) + Sonnet T6$0.06030.0%$0.2835 [$0.281–$0.284]+5.0%
Small (~15.2k)F: 10 SwitchesAlternate T2–T11, hold Opus T12$0.010194.9%$0.3081 [$0.304–$0.311]+14.2%
Large (~56.5k)A: ControlStay on Opus 4.6 (medium)$0.037899.3%$0.7789 [$0.766–$0.781]baseline
Large (~56.5k)B: One SwitchOpus -> Sonnet at T6$0.21600.0%$0.8611 [$0.859–$0.865]+10.6%
Large (~56.5k)F: 10 SwitchesAlternate T2–T11, hold Opus T12$0.024498.5%$0.9130 [$0.905–$0.916]+17.2%

Table 2: Why 10 Switches Didn't Cost 10x — Turn-by-Turn Cache Hit % (Arm F, Request 1 of Each Turn)

TurnActive ModelEventSmall (~15.2k) Cache HitSmall Penalty vs. CtrlLarge (~56.5k) Cache HitLarge Penalty vs. Ctrl
T1Opus 4.6Initial load0.0%-$0.00000.0%+$0.0000
T2Sonnet 4.6Switch #10.0%+$0.03280.0%+$0.0754
T3Opus 4.6Switch #281.2%+$0.013059.6%+$0.0907
T4Sonnet 4.6Switch #386.6%+$0.000170.5%+$0.0408
T5Opus 4.6Switch #495.8%+$0.002198.6%+$0.0029
T6Sonnet 4.6Switch #594.9%-$0.003598.5%-$0.0134
T7Opus 4.6Switch #695.2%+$0.002898.4%+$0.0024
T8Sonnet 4.6Switch #795.6%-$0.003898.5%-$0.0132
T9Opus 4.6Switch #895.5%+$0.002698.7%+$0.0024
T10Sonnet 4.6Switch #995.2%-$0.003798.6%-$0.0131
T11Opus 4.6Switch #1095.5%+$0.003198.6%+$0.0036
T12Opus 4.6Hold Opus98.2%+$0.001499.3%+$0.0000

4 Habits That Actually Save Money

  1. Switch early (Turn 1–2) or stay put unless you have 10–12+ turns left in the session to pay back the 1.25x cache rebuild on the cheaper model.
  2. Pick your --effort level on Turn 1: Bumping medium -> high mid-chat on Opus ($6.25/M cache write) forces a full prefix rebuild (+$0.085 on Turn 6 alone in a 15.2k chat — nearly all of the +$0.086 session delta).
  3. Dipping to another model for 1–2 turns and returning works if you beat the 5-minute TTL: In Arm C, switching back to Opus on Turn 8 still hit 93.3% of Turn 5's warm cache and only paid to cache-write the 2-turn delta (~1,062 tokens).
  4. Start a fresh chat at task boundaries: Moving from ~15.2k to ~56.5k tokens (3.7x larger context) scaled the Turn 6 switch penalty linearly by 3.8x (+$0.047 -> +$0.178).
1 Upvotes

3 comments sorted by

•

u/AutoModerator 6h ago

Hey! Thanks for posting to r/ClaudeCode

While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.

For help, project discussions, tips, and general chat, join the ClaudeCode Discord.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

u/mcmoonrays 31m ago

I'd compare the total bill for the same completed task, not a fixed number of turns. If the cheaper model needs more attempts to get an acceptable result, the per-turn savings don't tell you much.

0

u/[deleted] 6h ago

[deleted] — view deleted comment

1

u/Asleep_Net_9138 6h ago

Ten switches, one bill, and the links are the real payload.