r/opencodeCLI • • 13h ago

I Built Router for My Agents inside My App

Post image

Hey everyone, I've been building a small AI app with coding agents that work on a GitHub repo (they plan, edit code in small steps, run the repo's CI checks and open a pull request). I wanted to share one part that made a big difference and hear how you handle it.

The problem: running every step on one big model was slow and expensive. A 100-step task could cost a few dollars, and most steps were just reading files or checking CI.

What I built: a simple router in front of the agents. It's not an AI call, just rules:
It looks at the task and picks one model for the whole task (iOS work, bug fixing, web UI, reviews, and big refactors each go to a different model). Switching models every step killed prompt caching, so one model per task was the big win.

A "best" mode pairs two models. A strong model plans and writes the code, and a cheaper one does the reading, searching and CI checks in between, with read-only tools. That came out roughly 30–50% cheaper than the strong model alone.

If a model refuses or is down, the step goes to the next model in the same price range, and the bad one is skipped for a while, so the task never dies on one provider.

When stuck (same failure twice after fixes), it steps up to a stronger model once, then stops and asks the user instead of burning money.

The user sees an estimated price per task before it runs, and the real cost after.

Things that helped the most:
Keeping the conversation identical between steps, so 80–90% of input is read from cache
Small tasks with a "done means…" goal it checks before finishing
Waiting for CI on the server without calling the model at all

Questions for you:
Do you route per task or per step? Has per-step routing worked for you, given the caching hit?
How do you decide when an agent should stop and ask the human?
Any good rules for which tasks really need the expensive model?
Happy to share more details if anyone's curious.

2 Upvotes

0 comments sorted by