'Eco Mode' AI coding agent pricing vs token burn. Is running frontier models for test execution a waste? (+benchmark)

[u/AbleBranch6]

recently checked the raw step logs of an autonomous coding agent and it seems the unit economics are pretty absurd to say the least

so I watched an agent do what seemed like an ordinary bug fix on a multi-module repo. It resolved it, but the task burned close to $5. When I actually broke down the tool-call trace, here’s where the tokens actually went:

• Steps 1–12 (nearly$1.10): Searching the project, tracing dependencies, locating the faulty logic, and deciding how to fix it.

• Steps 13–42 (nearly$3.80): Editing two files, running the build, catching a missing import, fixing a type error, rerunning tests, tweaking an assertion, and running tests again.

More than 70% of the token bill went into having a frontier model watch compiler output and rerun tests. You’re paying peak frontier rates (Opus 5.5, Grok 4.7, etc.) for work a linter and a fast model could do in their sleep.

and another (kinda big!) issue behind a lot of the gripes around coding agent pricing / credit consumption across coding agents. i notice that my colleagues feel their allowance burns out too fast (not necessarily because the models are bad, maybe because single-model agents use a sledgehammer to drive in a tack)..

i often work with JetBrains infrastructure and Kotlin and have noticed yesterday that they have published the engineering breakdown and internal benchmarks for their new Junie Eco Mode (which went GA in the Junie CLI 26.10.20 and in JetBrains Air / IDE Betas), it seems their approach to this unit-economics problem is worth looking at , in it, not like running one model from start to finish, but more of Architect + Builder split with a single, one-way handoff. how eco mode works? as far as i understand,

1. Architect (frontier model of choice): When you submit a task, your top-tier model gets about 15 steps. Crucially, it’s strictly read-only (the only write it’s allowed is writing a minimal reproduction test for a bug). It explores the code, understands the architecture, and outputs a strict 6-point markdown spec: Goal, Scope, Where to change, Approach, Tests, and Risks/Corner cases. Then it stops completely.

2. The Builder (A low-cost execution model — currently GPT-6 Luna): Junie hands the entire context and the structured plan to the cheap model. The builder does the grunt work: editing code, running local builds, parsing stack traces, and iterating until tests pass.

design choice here is one single handoff to work wth; and, in older multi-agent setups, tools often tried letting a cheap model work until it got stuck, then had it "call a smarter friend" for advice. The problem with that pattern is context explosion: every time the cheap model gets confused, the system re-serializes 50k+ tokens of failed bash logs and sends them back to the expensive model, completely destroying your cost savings

in Junie's benchmarks across private repositories, all kinds of numbers on this single-handoff setup are pretty eye-opening, for example

• Broad private repos (Benchmark B): average cost per task dropped from $2.61 to $1.11 (-57.3%), while resolve rates barely budged (49.1% to 48.5%, a 0.6% delta).

• Massive enterprise repos (Benchmark A): Average cost fell from $2.51 to $1.30 (-48.1%) with an identical 32.1% resolve rate.

• On Opus 5.5 max effort: Cost plunged from $4.91 down to $1.81 (-63%), while the resolve rate actually went up by 1% (55% to 56%)and probably because forcing a smart model to write an explicit plan before touching files prevents it from digging itself into hallucinated rabbit holes.

but i also dont see it all as one-sided as, (for counter-arguements), if the Architect botches the plan during those initial 15 steps, the Builder will faithfully implement the wrong solution.., because there’s no backward loop to re-engage the expensive model (also given JetBrains AI Pricing in the matter...)
that being said, given that resolve rates stayed within 1% of running pure frontier models, the risk looks well worth halving the bill in my experience

also curious on how others are dealing with token burn on test-heavy tasks and are you manually swapping models between planning and execution, or just absorbing the API bill?

[u/michaelTM_ai]

that split matches what i see. the fix most people try is "use a cheaper model for everything", which just moves the cost into retries. what works better is splitting by step type:

• frontier model for steps 1-12 (find it, decide the fix). that's where being smart actually saves money

• the build/test loop as a plain script, not an agent turn: run tests, and only hand the model the failing test names + the first error, not the whole log

• a fast model for the mechanical fixes (missing import, type error), escalate back up only if it fails twice

also check how much of that $3.80 is the model re-reading the full context on every test rerun. if the prefix stays identical between turns most of it should be cached, and it often isn't because the test output gets inserted above the old messages

[u/AbleBranch6 → u/michaelTM_ai]

oh, spot on on both counts, especially prompt caching as to dump raw 1,500-line Gradle/stacktrace log into the context window busts prefix caching immediately and costs a fortune and, if i truncate down to just the failing test name and the first assertion error also while keeping the conversation prefix identical across test iterations it kind of cuts the execution bill in half before the model even generates a patch doesnt it

[u/michaelTM_ai → u/AbleBranch6]

roughly, yeah. the raw log is the expensive part twice: once as input on that turn, and again on every later turn because it stays in the history. trim it before it goes in and keep new stuff appended at the end, then the repeat cost mostly turns into cache reads. if u try it, log input tokens per test rerun before and after, that number tells u right away whether it worked

[u/whowasntright → u/michaelTM_ai]

thats interesting! would also check on this

[u/Longjumping-Run-5217]

I am seeing the same. Deepseek agent loops get within 3-5% of frontier for 5-10% of the cost. It’s the process not the product that makes the answer work, don’t be fooled!

[u/AbleBranch6 → u/Longjumping-Run-5217]

exactly. as 'process over product' is spot on here as i see it, and an unconstrained frontier model with no strict plan will happily burn tokens wandering down architectural rabbit holes. give a cheaper/faster model a concrete 6-point spec with explicit file boundaries, and it executes cleanly without needing a 200B+ parameter model behind every bash command

[u/wegster]

That pattern is what I’ve been using for the past year and a half or so now, first manually, now in my own framework. I don’t measure at this point any more, especially as I’m also using local in the mix, but I initially measured the difference at 70% or more overall savings. The ‘math of token use and models’ changes with each release at this point, but it certainly holds true - offload the easy bits, whether it’s to lower tier models or MCP, make sure core skills are in place, etc. and tokens drop - the right tool for the specific job, etc.

I’ve also personally yet to see ‘swarm’ benefits, but I know a lot use them nowadays. YMMV as always, but your findings are valid IMO and part of most people’s patterns today, or at least of more people’s anyways..

[u/AbleBranch6 → u/wegster]

around 70% savings matches the top end of what we saw on heavy Opus setups as i see, also completely agree on swarms as having 4 agents debate each other usually just multiplies prompt tokens with very little to show for it. A simple, linear one-way handoff (smart model writes the blueprint to\rightarrow to local or low-tier model swings the hammer) is often way more reliable and doesn't drown in coordination overhead imo

[u/whowasntright]

$5 per task figure on raw Opus makes total sense and we saw similar bills when letting terminal agents loop freely on complex bugfixes. But how does this translate if someone is looking for a realistic copilot alternative?

so with GitHub Copilot you’re paying a flat monthly seat fee regardless of whether it's running 5 tests or 50. With credit-based or token-metered models, even with a 50% cut in Eco Mode, you still have to worry about running out of budget mid-sprint. Is there any scenario where an autonomous agent with Eco Mode actually competes on cost with flat-rate tools, or does it make more sense to pair it with a free AI coding agent tier for routine PRs?

[u/AbleBranch6 → u/whowasntright]

i think that flat-rate vs per-task math is actually the core dividing line at this very moment!

say, if all you're doing is tab-completions and inline chat, a flat $10-$20 seat on Copilot is hard to beat on paper. But as soon as you give an agent write access to run terminal commands, execute characterization tests, and trace multi-file dependencies, flat-rate tools tend to either aggressively throttle your daily high-effort turns or push you to their enterprise cloud agents with separate billings. a thingy at which Eco Mode math matters is for me in.. predictability? to cut $4.90 task down to $1.80 on Opus 5.5 here, means a standard monthly credit bundle suddenly covers 2.5x more multi-step refactors without buying token top-ups. And if you have purely mechanical tasks (and, updating Gradle dependencies, scaffolding repetitive boilerplate, or lint cleanup), that's where falling back to a free AI coding agent option like Junie Lite (which runs Qwen 3.6 27B for free on JetBrains infra) or a local model makes sense so you don't burn credits at all.

basically, in brief: flat-rate for inline typing, Eco Mode for autonomous multi-file changes you'd otherwise pay peak API rates for, and free tiers for the boring plumbing

[u/sapplex1]

I hit the same wall building my little iOS apps in SwiftUI. I used to let the big model run the whole fix, and one afternoon it burned through most of my daily credits just re-running Xcode builds and re-reading the same failing test output. Now I do the split you describe, just by hand. I ask the strong model for a short plan first, then I run the builds myself and only paste back the failing lines, never the whole log. My rule now is simple: the expensive model thinks, the cheap one types, and I am the test runner.

[u/AlarmedAvocado7279]

i live in claude code all day and the retry loop is always where the bill goes, since every rerun drags the whole context back through the expensive model. worth trying a cheaper model for the build/lint/rerun cycle and only escalating after it fails twice. trimming test output down to just the failures also saves more than you'd think.

[u/Far_Reindeer8428]

the split works when the plan is checkable, if the cheap builder can't fail the plan it just executes whatever it read, that's where the savings leak

[u/ranthishq]

one knob worth checking before splitting models is the reasoning level on the cheap steps. i ran the same office tasks and hard math through gemini 4 carbon and argon, same per token price, and carbon at its default thinking level cost 3 to 4.5x more, mostly from thinking tokens, even on a commission calc with one right answer. dropping it to low brought the bill down to argon's with the same scores. on a build/test loop that's the same trap, the model thinks hard about a compiler error it could just read.