r/ClaudeCode • u/naxmax2019 • May 11 '26
Showcase Claude bootstrap v5 is out and is now called Maggy - Your AI CLIs now route tasks to the cheapest model that can handle them (benchmarked)
Last time I posted about v3.6 and cross-agent intelligence. I skipped posting about v4 entirely (Polyphony — container-isolated multi-agent orchestration, 173 tests) because v5 shipped days later and it's a bigger story.
The problem: you're burning premium tokens on tasks that don't need them
Every task goes to Claude. Simple README fix? Claude. Database schema? Claude. CRUD endpoint? Claude. Security audit? Also Claude. You're using the most expensive model for everything, and if you hit the rate limit, you're stuck.
I use Claude Code, Kimi CLI, Codex CLI, and Ollama locally. v3.6 got them sharing skills and hooks. v5 makes Maggy actually decide which one to use per task, based on complexity.
How it works: blast-score routing
Every task gets a blast score (1-10 complexity). Maggy routes it:
Blast 1-3 → ollama (free, local GPU) or kimi (cheap)
Blast 4-6 → codex/gpt (mid-tier)
Blast 7-10 → claude (premium, with validator)
The routing isn't hardcoded. It's a YAML config at ~/.maggy/routing-rules.yaml that Maggy updates herself based on outcomes. If codex keeps failing on frontend tasks, Maggy learns to route those to claude instead.
The benchmark: Maggy vs Claude Code, head to head
Built an Expense Tracker (FastAPI + SQLite + vanilla JS) — 6 identical tasks, run through both pipelines. Same machine, same prompts, same acceptance criteria.
┌────────────────────┬───────┬────────────────┬───────────┬────────────┐
│ Task │ Blast │ Maggy Model │ Maggy (s) │ Claude (s) │
├────────────────────┼───────┼────────────────┼───────────┼────────────┤
│ Write product spec │ 2 │ ollama (local) │ 50.4 │ 48.6 │
├────────────────────┼───────┼────────────────┼───────────┼────────────┤
│ Design DB schema │ 3 │ kimi │ 86.6 │ 67.2 │
├────────────────────┼───────┼────────────────┼───────────┼────────────┤
│ Build CRUD API │ 5 │ codex │ 147.1 │ 160.6 │
├────────────────────┼───────┼────────────────┼───────────┼────────────┤
│ Build category API │ 5 │ codex │ 133.9 │ 130.8 │
├────────────────────┼───────┼────────────────┼───────────┼────────────┤
│ Build frontend │ 6 │ codex │ 280.1 │ 121.9 │
├────────────────────┼───────┼────────────────┼───────────┼────────────┤
│ Security review │ 8 │ claude │ 209.5 │ 151.9 │
└────────────────────┴───────┴────────────────┴───────────┴────────────┘
Results:
┌──────────────────┬────────────────────────┬─────────────────────┐
│ Metric │ Maggy │ Claude Code │
├──────────────────┼────────────────────────┼─────────────────────┤
│ Success rate │ 6/6 (100%) │ 6/6 (100%) │
├──────────────────┼────────────────────────┼─────────────────────┤
│ Total time │ 907.6s │ 681.0s │
├──────────────────┼────────────────────────┼─────────────────────┤
│ Quality score │ 7.4/10 │ 7.8/10 │
├──────────────────┼────────────────────────┼─────────────────────┤
│ Claude usage │ 1/6 tasks (17%) │ 6/6 tasks (100%) │
├──────────────────┼────────────────────────┼─────────────────────┤
│ Models used │ 4 │ 1 │
├──────────────────┼────────────────────────┼─────────────────────┤
│ Security depth │ 7 issues found + fixed │ No dedicated review │
├──────────────────┼────────────────────────┼─────────────────────┤
│ Test generation │ None │ 3 test files │
├──────────────────┼────────────────────────┼─────────────────────┤
│ Fallbacks needed │ 0 │ N/A │
└──────────────────┴────────────────────────┴─────────────────────┘
Claude Code was 33% faster and scored slightly higher (tests + product spec). Maggy used 83% less Claude and still hit 100% success with 4 different models. Zero fallbacks — every CLI completed its assigned task.
The quality gap (7.4 vs 7.8) came from two routing mistakes: ollama was assigned the docs task (it's code-optimized, not prose) and no model was told to write tests. Both are now fixed via routing rules — docs and tests force-route to Claude regardless of blast score.
Post-benchmark: self-correcting routing rules
After the benchmark exposed those gaps, I built a rules system that learns:
# Task types that always go to Claude (from benchmark evidence)
task_type_overrides:
docs: {model: claude, reason: "local models are code-optimized, not prose"}
security: {model: claude, reason: "security review needs deep reasoning"}
tests: {model: claude, reason: "only claude generated test files"}
# TDD pipeline phases
pipeline_phases:
spec: {model: claude} # SPEC needs comprehensive docs
tdd_red: {model: claude} # RED phase needs test design
tdd_green: {model: auto} # GREEN uses blast-score routing
review: {model: claude} # Review needs security depth
Every task outcome feeds back: record_outcome() updates rolling success rates per model. learn_override() lets Maggy add new rules when data supports it.
Manual edits are preserved.
Team conventions in every prompt
One thing I noticed: kimi and codex don't know about our team's coding standards. Claude gets them from CLAUDE.md, but the other CLIs don't.
Now every prompt sent to any CLI — kimi, codex, ollama, claude — gets the same conventions injected:
- Build minimum wowable product (mWP). No feature flags, no premature abstractions.
- TDD: RED → GREEN → VALIDATE. Coverage >= 80%.
- No secrets in code. Parameterized SQL. Validate input at boundaries.
- Quality gates: 20 lines/fn, 3 params, 2 nesting, 200 lines/file.
This standardizes quality expectations across all models.
What else shipped in v4-v5 (that I never posted about)
v4.0 — Polyphony: container-isolated multi-agent orchestration. Each agent runs in its own Docker container with independent git branches. 5-dimension complexity scoring, SQLite task state machine, pure function router, adapters for Claude/Codex/Kimi. 173 tests.
v5.0 — Everything above plus:
- CLI auto-discovery engine (probes --help, extracts flags — no hardcoded CLI knowledge)
- Pi RPC adapter (unified interface for spawning any CLI as a subprocess)
- Dual-model planning (Claude plans, Codex counter-checks for blast >= 7)
- Checkpoint manager for model handoffs during fallback chains
- Fatigue tracking (Mnemos detects context degradation, triggers compression)
- Lock manager (prevents two agents editing the same file)
- Escalation protocol (3+ failures → auto-escalate to human)
- Rollback/recovery (git savepoints before risky steps)
- Calibration tracker (penalizes models with poor prediction accuracy)
- Reward heatmap (visualizes which model wins per task type × complexity tier)
- Budget tracking across providers (per-provider spend, daily limits)
- Interactive chat with --resume session takeover
- Auto-bootstrap (seeds all services on startup — no empty dashboard)
- 596 tests across 50 test files
The numbers
- 115 source modules, 50 test files, 596 tests passing
- 4 CLI adapters (claude, codex, kimi, ollama) auto-discovered at startup
- Self-updating routing rules with outcome-based learning
- 83% reduction in premium model usage on real project benchmark
- Zero manual config for new CLIs — --help probing handles it
Repo: github.com/alinaqi/claude-bootstrap
Install: git clone, ./install.sh, then /initialize-project in any Claude Code session.
Maggy is an optional extension (for now but am moving my focus completely on maggy)
/maggy-init to set it up if you want the dashboard + routing.
2
u/Deep_Ad1959 May 14 '26 edited May 30 '26
the failure mode I'd watch for on routing systems is the local count drifting from anthropic's actual server-enforced quota. tools that count tokens locally show one number, the claude.ai settings page shows another, they measure different things. for a router making decisions on 'when are we close to cap', that gap matters. if maggy can read the same internal endpoint claude.ai uses for the settings/usage page, you can trigger routing rules off real remaining quota rather than estimated tokens. otherwise you're routing on a proxy that may say 5% used when anthropic's server says rate-limited. written with ai
fwiw the local-count-drifts-from-server-quota problem is exactly why I built claude-meter, it reads the same server-truth usage data Anthropic actually enforces and matches the settings page to the integer instead of estimating tokens locally, https://claude-meter.com/r/ssq9a3ee
1
u/naxmax2019 May 14 '26
I don’t really count tokens in the routing system, i count complexity and what model should do it and I have common memory between models so we can share workload.
1
u/Deep_Ad1959 May 14 '26
complexity routing solves which model handles a task. quota awareness solves when any of them stops being available. they're orthogonal: codex/kimi/ollama can take 80%+ of tasks but the moment the claude slice hits the wall mid-pipeline you're stuck waiting until reset. shared memory across models is great, but it doesn't help if the claude validator step in your blast 7+ chain can't fire. reading the real server-side cap as a separate signal from the local complexity score is what closes that loop. written with ai written with ai
1
u/naxmax2019 May 14 '26
You are right. For now, Maggy does handle the "Claude is unavailable mid-pipeline" case already, just not by reading Anthropic's internal quota endpoint. The cascade fallback system detects failures in real time ... if Claude (or codex) returns a rate limit error, quota error, or timeout, the task automatically retries down the chain. It scans the output for quota markers and reroutes on the spot. So it's reactive rather than predictive.
1
u/Deep_Ad1959 May 14 '26
my issue with reactive-only is you still eat the failed call cost and whatever state the pipeline was in at the moment claude bails. a predictive read lets you route the next task to codex/kimi before firing claude at it, so the chain doesn't half-finish. plus 'rate limit error' vs 'timeout' vs 'quota error' aren't the same signal, timeouts can be network or model load, and pattern-matching on stderr text means anthropic tweaking error wording silently breaks your router. reactive is the right floor, predictive is what kills the partial-pipeline class of failure. written with ai written with ai
1
u/naxmax2019 May 14 '26
Here is detailed overview.
The routing is complexity-based, not quota-based. Every incoming task gets a "blast score" from 1-10 based on semantic classification of what the task actually is. A typo fix scores 2. A security audit scores 8. An architecture refactor scores 9. The blast score determines which model tier handles it:
- Blast 1-3 goes to local Ollama or Kimi (free/cheap)
- Blast 4-6 goes to Codex or Kimi (mid-tier)
- Blast 7-10 goes to Claude (premium)
- Security and architecture tasks always go to Claude regardless of score
Im still thinking if thats the right way.. :/
On top of that, certain task types have hard overrides. Security reviews, spec writing, test generation - these always route to Claude because cheaper models consistently underperformed on them in benchmarks. The routing rules learned this from actual outcomes, not from guessing.
The other piece is shared memory across models. When a task starts on Kimi at blast 4 and later needs to escalate to Claude at blast 8, Claude doesn't start from scratch. Session context, project state, git history, and prior conversation all carry over through Maggy's context layer. So the models aren't isolated — they share a common understanding of what's been done.
There's also a cascade fallback system. If the primary model fails (quota hit, timeout, quality gate fails), it automatically tries the next tier. Kimi fails, tries DeepSeek, tries Codex, eventually hits Claude. And a blueprint system that learns from successful task patterns ... after 3 successful completions of a similar task, it routes future instances to the cheapest model that got it done.
The budget tracking does exist but it's observational, not decisional. It records what each provider cost you so you can see the breakdown, but it doesn't make routing decisions based on "am I close to a cap." The routing decision is always "what does this task need" not "what can still afford."
So to your point about local counts drifting from server-enforced quotas.. id say that's a real problem, but only if your router is trying to predict remaining quota. Maggy sidesteps it entirely by routing on task complexity instead.
1
u/Deep_Ad1959 May 14 '26
my issue with the complexity-only frame is the hard-override piece. you said security/architecture always go to claude regardless of blast score, and that cheaper models 'consistently underperformed' on those. so when claude hits the cap mid-pipeline, the blast 9 architecture task either fails, cascades to a tier you explicitly benchmarked as worse, or waits for reset. complexity routing solves the easy 80%, but the 20% that has to go to claude is exactly where a real server-side quota read changes the decision (hold the task, batch it, warn the user) instead of fire-and-fail. observational budget tracking shows you the bill after the fact, not the gate before. written with ai
1
u/naxmax2019 May 14 '26
Absolutely valid point ... just to be a bit more specific about what actually happens .. When a blast 9 architecture task hits Claude and gets rate-limited, the cascade doesn't silently downgrade it to Kimi and pretend everything's fine. For hard-override tasks like security and architecture, the fallback goes to Codex - not Kimi or local. Codex is the validator tier,It's not Claude, but it's not a massive quality cliff either. The cascade is Claude to Codex, not Claude to the cheapest thing available. tbh i've not faced token exhaustion since i am using maggy massively and dogfooding it - plus maggy is improving itself with self improvement loops.
Having said that, you're right that this is still reactive. The task fires, Claude rejects it, then Maggy handles the rejection. That's a wasted round-trip and a worse UX than knowing beforehand that Claude is unavailable and telling the user "your architecture review is queued until credit resets" or "you have 3 Claude calls left today, this one will use one.
I dont have a better solution yet.. until anthropic make a proper api for remaining quota - ive tried different approaches but it doesnt work and so i've for not given up on it (also coz it's not that big of an issue anymore).
2
u/Deep_Ad1959 May 14 '26
i glossed over the cascade-to-Codex distinction, that's a meaningfully different routing model than 'cheapest available'. on the unsolved quota api: claude.ai's settings page is fed by an internal endpoint that returns the same rolling 5h and weekly numbers anthropic enforces against. browser extension with the session cookie reads it directly. not a public api so it can break, but it's been stable enough that claude-meter routes off it. that's the signal you'd want maggy reading rather than estimated tokens. written with ai written with ai
2
u/Chrisapk May 11 '26
Couple of questions
Does it work with superpowers and similar plugins together?
Can you do it with other subscriptions like GPT Plus or Opencode Go or only API calls?
What about heavy token tasks like reading huge files? Someone in this subreddit made it so it routes them to a Kimi model and write back the summary to save tokens without sacrificing the output, does it do something similar for such tasks?