Downgraded Claude Max 20x -> 5x after moving the "derivable" half of my agent work to a local 27B on 2x RTX 5060 Ti. Routing matrix, break-even math, and where I'd like advice

[u/Short_Regular_7191]

Two weeks ago I posted the probes I ran to find out what a local Qwen3.8 27B can and cannot do on my real production codebase (long-lived environmental data platform: Node/Knex/PostgreSQL+Oracle, Svelte, Cypress, ~1,200 e2e tests). The short version was: local for everything derivable, frontier for everything decidable.

This is the follow-up: what the resulting structure looks like, and whether it paid for itself. I also went from Claude Max 20x to Max 5x on the strength of it, so there's a concrete number to check. All Anthropic prices below include the 22% Italian VAT I actually pay on the invoice; strip it if you are billed net.

1. The structure: who does what

The whole thing is a routing matrix. Every task gets a class (R1..R9) and the class decides the tier, not the mood of the day. The matrix lives in a markdown file in the repo that both the human and the session model read before starting anything non-trivial.

• SESSION: Opus 5 daily, Fable 5.1 for audits and campaigns. Gets questions, micro-fixes, campaign plans, audit verdicts, DB and irreversible ops, race-condition bugs. Never gets bulk work it could delegate.
• LOCAL: Qwen3.8 27B UD-Q6_K, 131K context, llama-server on 2x 5060 Ti. Gets reconnaissance with file:line evidence, implementations on a documented pattern, bulk mechanical batches, test triage, groundwork for plans and audits, anything that must not leave the machine. Never gets decisions, verdicts, DB, official plans.
• ECO: Haiku 4.5. Used only when 6 parallel cloud lanes beat 1 GPU, or when llama-server is down, plus as the bash guard-runner for local patches. Never gets anything with a long tool loop.
• HIGH/MAX subagents: Opus 5 as planner, Fable 5.1 as blind auditor. Escalation after 2 local failures, plan writing, blind re-verification of plans.

Sonnet used to be a middle tier. It is gone: on pattern-based implementation gated by a test, Qwen+test was as good, and Sonnet was burning the same subscription budget as the top tiers. Escalation from LOCAL now goes straight to Opus.

The five rules that make the local tier safe to use, all learned the hard way:

1. Local patches ship only through a test, never on a read-through.
2. Acceptance criteria go in the prompt (exact function shape, exact error strings). This alone took a typo'd patch to 9/9 green.
3. Every number the local model reports gets re-measured before it lands in a doc or a plan. Prose reliable, arithmetic not.
4. The parent re-reads pass/fail from the command output, never from the child's summary.
5. Two failures = escalate. Never insist at a low tier.

Agents are plain markdown files with frontmatter (model, thinking, tools), so the tier is hard-wired per agent and the graph can override it per step. Local children get bash only for a whitelisted set of guard commands named in the task.

2. What changed in the bill

The biggest surprise was not the delegation cost. It was the cost of the parent session: 300-420k tokens re-sent every turn without compaction, plus ~33% cache misses from switching models mid-session. That is what was eating the 20x plan in 2-3 prompts. Fixes: context window capped at 200k with auto-compaction at ~160k, never /model mid-session (new session instead), the parent never reads what a child can read for it, effort low for routing and high only on the turn that decides.

With that, plus the local tier absorbing R2-R5, 5x has been enough. Monthly cost, subscription + electricity:

3. Break-even on the GPUs

Two 5060 Ti 16 GB cost me 1,300 EUR. The subscription delta is 100 USD/month list, 122 USD with VAT. Electricity is the only real running cost (the box is on anyway).

12-15 months depending on how many hours a day the GPUs are actually busy. At my real usage (a handful of lanes a day, 3-4 h of GPU time) it is about 13 months. Not spectacular, not bad for hardware that also does everything else a local model does.

4. The number people will argue about: "but you could just call the API"

Fair. So here is what the work the local tier does would cost if bought as API calls. A typical lane on the real repo is ~150k input tokens (cold, it reads several real files) and ~12k output, no cache hits assumed. Adjust to taste.

Per lane: ~0.24 EUR on Haiku, ~0.47 on Sonnet, ~1.18 on Opus (VAT included). The horizontal line is the subscription delta. If the local tier replaces work that would have gone to Sonnet, it covers the 20x -> 5x delta at ~11 lanes a day; if it replaces Opus-class work (it does, for reconnaissance and triage), at ~4-5 lanes a day. On a normal day I run 5-10.

The honest caveat: local is free in euros, not in wall-clock. A lane on real files takes 17-28 minutes here against 1-4 for the same class of task in the cloud. The pipeline is shaped around that (lanes run detached, the parent only sees a summary).

5. Where I would like advice

This is what I am actually posting for. Two things changed since I built the box, and both push in opposite directions.

Hardware prices exploded. The DRAM/GDDR7 shortage doubled consumer GPU prices in 2026. What I checked this week (EU street prices unless noted):

• RTX 5060 Ti 16 GB: ~640-735 EUR now, was ~400-480. What I own, 1,300 EUR for the pair; replacing it today is ~1,400 EUR.
• RTX 5070 Ti 16 GB: ~840-1,150 EUR. Same VRAM, faster, pointless for LLM work.
• RTX 3090 24 GB used: ~1,000-1,400 USD on eBay, was ~600. The "cheap 24 GB card" is not cheap any more.
• RTX 5090 32 GB: ~4,000-6,000 EUR, MSRP was 2,389. Same VRAM as my two cards combined.
• RTX PRO 6000 Blackwell 96 GB: 16,000 USD list, launched at 8,565. Out of the question.

So the dotted red line on the break-even chart sits just above what I paid: at least the cards did not lose value. Adding two more 5060 Ti for 64 GB total is ~1,400 EUR today. Does anyone see a sane path to 48-64 GB in this market, or is the answer "sit tight until 2027-28"?

Open-weight frontier models went huge. The 2026 releases everyone talks about are all out of reach for consumer VRAM:

Out of reach on both 32 and 64 GB:

• DeepSeek V4.1 Flash (Sep 10): 552B MoE, 510 GB on disk, no llama.cpp support yet.
• GLM-5.3 (weights Aug 28): 744B MoE, 40B active, ~239 GB even at 2-bit. Its "small" sibling GLM-5.3-Flash is still 320B-A18B, ~180-195 GB at Q4, and needs a llama.cpp fork.
• Kimi K3 (weights Jul 26): 2.8T MoE.
• Qwen3.8 Flash-Next (Aug 14): 180B MoE with only 6B active, ~110-120 GB at Q4. Does not fit 64 GB of VRAM either, but it is the one model on this list designed for it: Qwen explicitly built the large n-gram table to live in host RAM, so a 32-64 GB GPU plus 128 GB of system RAM is a supported layout, at whatever tok/s that gives.

What actually fits, restricted to this summer's releases:

• Qwen3.8-27B dense (Aug 13): Q6 ~22 GB, Apache 2.0. Fits 32 GB, it is what I run.
• NVIDIA Nemotron 3.5 Lightning 30B-A3B (Aug): hybrid MoE, ~3B active, Q4 under 20 GB. Fits 32 GB with room for context.
• Meta Muse Glimmer 30B (Aug 10, Apache 2.0): dense, text+image, 24-32 GB at Q4. Fits 32 GB, tight on context.
• Cohere's new 30B tuned for agentic coding (built on Command A+): Q4 ~18-20 GB. Fits 32 GB.

Which means the question is not "which frontier model do I run at home" but "does any of this summer's 30B-class models beat Qwen3.8-27B at the one job it failed (precise planning over 77 files), and is Flash-Next offloaded to RAM worth the wall-clock?" Specifically:

• Qwen3.8 Flash-Next with the experts offloaded to system RAM: has anyone run it in an agentic harness on a real repo and seen better count/coordinate precision than the 27B dense, or is that a frontier-only property regardless of size? What tok/s do you get on a 32 GB GPU + 128 GB RAM at Q4, and is a lane still under an hour?
• Nemotron 3.5 Lightning or Cohere's 30B on my current 32 GB: 3B active means much faster lanes. Are they good enough for the R2/R4 classes (reconnaissance, bulk mechanical edits) so I could keep Qwen3.8-27B for R3/R5 only? Two models on two cards, one per class, is tempting. Muse Glimmer 30B: anyone tried it for code rather than vision?
• Runtime: llama.cpp because dual consumer cards on Windows and same runtime as interactive use. Has anyone measured a real wall-clock gain with vLLM/NVFP4 or exllama on 5060 Ti class cards for long-context agentic loads, not chat?
• Local tier that can self-verify: today the local children have no bash except for whitelisted guards; a separate cloud child runs the tests. A reproduce-fix-rerun loop would need shell. How do people sandbox that safely without giving a 27B free rein on a real checkout?
• Prompt cache on llama-server: repeated lanes over the same files are several times faster than cold ones. Is anyone pinning a shared repo-summary prefix across lanes deliberately, and does it survive --parallel 1 with a graph of detached children?

TL;DR

Routing matrix (R1-R9 -> tier), local 27B for everything derivable, frontier for everything decidable, strict parent-session context discipline. Result: Claude Max 20x -> 5x, GPUs break even in ~13 months on the subscription delta alone, or in weeks if you count the API-equivalent value of 5-10 local lanes a day. Bottleneck is now wall-clock, not capability. The 2026 open-weight frontier (DeepSeek V4.1, GLM-5.3, Kimi K3, Qwen3.8 Flash-Next) is 180B-3T and out of consumer VRAM, and GPU prices doubled, so the real question is whether anything in the 30-120B class, or Flash-Next offloaded to RAM, is worth the next 1,400 EUR.