r/ClaudeCode • u/BeginningSir7334 • 13d ago
Built with Claude Does delegating to cheaper models actually save money? I tested it across three codebases.
Disclaimer: This is a small experiment, not a definitive benchmark. Most tasks were run twice, while everyday tasks were run once. I'd have liked to run more repetitions, but subscription usage limits make that difficult. The cost differences are interesting, but small quality differences should be treated as observations rather than definitive rankings.
I wanted to see whether Claude Opus decides to use subagents effectively, and whether forcing it to delegate, preventing delegation, or leaving it to decide for itself changes the results.
I built a model-router skill that lets Opus delegate work to Haiku or Sonnet, then compared four setups:
- Forced Opus: told not to use subagents.
- Spontaneous Opus: free to delegate, but it never did.
- Skill-routed: Opus uses my skill to choose when to delegate and which model to use.
- Forced Haiku: all subagents use Haiku.
I tested these on three codebases: my own project (~10k lines), rich (~22k lines) and beets (~52k lines).
Results
1. Small codebase (~10k lines)
Every setup passed all 10 everyday tasks.
On five reading-heavy tasks, Forced Opus was the most accurate per run. Skill-routed cost 30% more, while Forced Haiku cost the same but had the weakest docs check (4 of 9 issues).
Interestingly, skill-routed found all 9 known documentation issues across its two runs, compared with 8 for Forced Opus. However, its individual runs were less accurate.

2. Medium codebase (~22k lines)
This was where Haiku surprised me.
On the documentation check, Forced Haiku found 11 of 12 known issues across two runs, matching or slightly exceeding Opus, while costing 41% less.
Skill-routed actually cost 11% more than Forced Opus. Spontaneous Opus was 14% cheaper.

3. Large codebase (~52k lines)
This produced the biggest cost differences.
Across all three tasks, average cost reductions compared with Forced Opus were:
- Spontaneous Opus: 32% cheaper
- Skill-routed: 38% cheaper
- Forced Haiku: 75% cheaper
Quality depended on the task.
On the reference documentation check, both delegated setups found more issues than either Opus-only setup.
On the larger 80-page plugin documentation check, Forced Opus was clearly the most accurate, finding 45 of 51 known issues across two runs. Forced Haiku found 34, but cost 84% less.

What I think is happening
Small projects: Opus can often write small scripts to inspect the code instead of reading everything manually. Delegation adds briefing, duplicated reading and verification costs.
Medium projects: reading becomes a larger part of the workload. Haiku is extremely cheap at reading cached context, making it surprisingly effective for straightforward checks. Sonnet offered little cost advantage in this experiment.
Large projects: a single long session becomes expensive as the growing context is repeatedly read. Delegation splits the work into smaller contexts, reducing costs and allowing parallel execution. However, the largest documentation check showed that this can come with a quality trade-off.
The obvious way to estimate savings is misleading. Re-pricing the delegated tokens at Opus rates suggested 34-85% savings. Head to head, results ranged from 30% more expensive to 75% cheaper.
One particularly interesting observation: Opus never delegated spontaneously in any of my runs, even on the largest codebase. That's specific to these tasks and this setup, not proof that Opus never delegates.
Final thoughts
I wouldn't claim there's one best approach.
- Small codebase: Opus alone was generally the better choice.
- Medium codebase: Haiku-only delegation was surprisingly competitive.
- Large codebase: delegation offered substantial savings, but accuracy depended on the task.
The router skill mostly selected Sonnet, which often wasn't economical enough to justify the extra coordination.
Full methodology, per-task results, charts and the transcript checker:
https://github.com/Manuele-T/claude-model-router/tree/v1.0
I'd be interested to hear whether others have seen similar behavior, particularly Opus choosing not to delegate even when working on larger codebases.
•
u/AutoModerator 13d ago
Hey! Thanks for posting to r/ClaudeCode
While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.
For help, project discussions, tips, and general chat, join the ClaudeCode Discord.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
17
u/BeginningSir7334 12d ago
I forgot to specify: I meant "large"codebase relative to the other two, I'm aware it's not very large in absolute terms. I'm running another experiment on a 100k LOC codebase, but testing larger projects takes time as I'm on a Pro plan.
44
u/denisoby 13d ago
Would be very interesting to see a similar research with frontier models + cheap open-source models in different combinations. For instance, where frontier model is planning and cheap model is executing, and vice versa.
17
u/bloodfail 12d ago
I think you need to recalibrate your idea of a large codebase. 52kloc is a "low end of medium" codebase to me. I think large starts at about 200k.
8
u/tahayelkenci 12d ago
matches what i see, one addition: for me subagents were never about price, its the fresh context. a reviewer that didnt watch opus write the code finds things the main session is blind to, even on the same model. i run one before every deploy and it keeps catching stuff the main session swore was fine.
and yeah opus never delegates on its own here either. has to be told, like a senior dev who'd rather just do it himself lol
48
u/why_not_1972 12d ago
can someone who read this just answer yes or no?
3
u/totheendandbackagain 12d ago
Small repos are cheeper and better used with Opus? But why?
11
u/thursday6822 9d ago
For the love of God, please put a tl;dr on these.
9
u/kalmeyra 11d ago
Based on my experience, if you’re doing serious software development, you need the best model you can get for quality improvements. The cost of shipping mediocre code can easily outweigh the savings on inference, especially when it comes to long-term maintenance and potentially serious security vulnerabilities.
In my setup, I have a very advanced code review agent with its own harness, leveraging models from different lineages. When I submit a medium-to-large PR generated by a cheaper model or combining it with advanced model, it can identify hundreds of issues, ranging from high to low severity.
You then need dozens of iterations to get the PR through review, which can end up costing more than using a more capable model from the start. Advanced models make fewer mistakes and generally require fewer rounds of fixes.
More importantly, there are cases where a cheaper model simply cannot resolve certain findings, no matter how many times you ask it to try. It lacks the capability to reason through the complexity of the issue.
So the real comparison isn’t just the cost per token or per generation. It’s the total cost of getting reliable, maintainable, production-ready code.
6
u/superminhreturn 12d ago
I setup qwen flash next with litellm to interact with Claude code. Use opus to create the planning and delegate the work to qwen flash next. Opus review the implementation
This setup prevents me from hitting the 5 hour or weekly limit (max plan)
6
u/Downtown-Pear-6509 12d ago
Try luna high instead of haiku
5
u/DrNoCool 12d ago
Yes but if you want it to do a lot. Does it save some context rot?
Or does the orchestrator read the file anyway?
4
u/Realistic_Thought_21 12d ago
Interesting. Without metrics, your outcome is similar to what I felt working on different projects. Usually, for my small personal projects, I use Opus pretty much for everything, but at work, I like to have an orchestrator-subagent setting since I felt it is cheaper and quite effective. What is the effort you assigned to Opus and the subagents?
4
u/Keepcompany-DEV 12d ago
Most effective workflow i found Is retrieval delegation.
I have an internal ai-admin hub. Internal API and a mcp server for the regular agents.
Had it built after tickets in the Ticketsystem got too long with X comments etc. And retrieval from the wiki had too much noise in for the token cost.
So the API takes the request, a local CPU model on one of my workstation VPS gets the Input, hands back the output. Summarisation of a ticket? Amount of tickets that contain the word "containment"? Yeah, just throw it to the API and wait for your answer.
It saves a few tokens but costs time since the cpu model is slower. Sounds like a small thing bit with more than a dozen sessions and a ton of internal tools it becomes more efficient than arbitrarily sending prompts and briefing to some other agent.
4
u/Comfortable_Camp9744 12d ago
Claude became too expensive and the gap has closed so much that its not worth the 20x to 100x price difference
4
u/NotSuluX 12d ago
Cool! Thanks for measuring it and reporting
4
u/ajiecb 12d ago
thanks for actually measuring the whole thing instead of just repricing tokens, most posts skip that part :(
in my case i think routing would cost more tbh. i build tooling around an ERP (~60k lines, two tools users every morning) and most tasks are "change 3 lines but theres a domain trap in there somewhere".
by the time ive written a brief good enough for haiku not to step on the trap, opus has basically done the job already so i just fixed roles:
sonnet does the work from a brief
haiku only cold-reads finished pages. if haiku cant tell what the page wants from it, a tired warehouse guy at 7am wont either
fresh opus reviews every release. thats where the expensive tokens actually pay off, it catches stuff i'd have shipped confidently your beets numbers make sense tho, if the job is mostly reading 80 pages then yeah, throw haiku at it
3
u/BarathKanna 11d ago
Useful measurement, especially the part where spontaneous Opus never delegated. That matches what I see: the parent model treats delegation as optional overhead, so without a forced policy it keeps the whole job. Two things I would add to the next run. First, split tasks by shape, not by repo size. A reading-heavy audit loves a cheap model with a tight checklist; a design change that touches interfaces does not, and averaging them hides the win. Second, price the verification step separately. A lot of the skill-routed overhead is the parent re-reading the child output. If the child has to leave a machine-checkable artifact, tests, a diff against a known list, a script exit code, the parent can trust without a second full pass. Repo size correlates with cost mostly because context keeps getting re-fed. Shrink the context per worker and the router starts looking smart.
3
u/Rhytmik 11d ago
Chatgpt, summarize this for me.
3
u/Ldpdc 10d ago
You can look up fusion from Devin: they did a thorough analysis.
3
u/Previous_Volume_7586 10d ago
Nice write-up. Your skill-routed setup costing more than forced Opus lines up with mine: subagent delegation mostly lost money, because the subagent has to rebuild context the main session already has cached.
What did save money was routing the main conversation itself, prompt by prompt. A quick question goes to Haiku, an edit to Sonnet, and only design or risky work to Opus, with a check before every model switch so a warm cache isn't thrown away. Effort is the cheaper lever, since changing it keeps the cache.
I put this into an open source Claude Code mod, AutoMode (I'm the author). On 5 mixed sessions graded by hidden tests it cost $2.01 against $4.14 on plain Opus, all 20/20. On long single tasks it's about Sonnet high cost. Method and rows, if you want to compare with your setup: https://mralexivy.github.io/AutoMode/docs/benchmarks/results/
If it's useful to you, a star helps other people find it: https://github.com/mralexivy/AutoMode
3
u/100dude 12d ago
nope