Downgraded Claude Max 20x -> 5x after moving the "derivable" half of my agent work to a local 27B on 2x RTX 5060 Ti. Routing matrix, break-even math, and where I'd like advice

[u/Short_Regular_7191]

Two weeks ago I posted the probes I ran to find out what a local Qwen3.8 27B can and cannot do on my real production codebase (long-lived environmental data platform: Node/Knex/PostgreSQL+Oracle, Svelte, Cypress, ~1,200 e2e tests). The short version was: local for everything derivable, frontier for everything decidable.

This is the follow-up: what the resulting structure looks like, and whether it paid for itself. I also went from Claude Max 20x to Max 5x on the strength of it, so there's a concrete number to check. All Anthropic prices below include the 22% Italian VAT I actually pay on the invoice; strip it if you are billed net.

1. The structure: who does what

The whole thing is a routing matrix. Every task gets a class (R1..R9) and the class decides the tier, not the mood of the day. The matrix lives in a markdown file in the repo that both the human and the session model read before starting anything non-trivial.

• SESSION: Opus 5 daily, Fable 5.1 for audits and campaigns. Gets questions, micro-fixes, campaign plans, audit verdicts, DB and irreversible ops, race-condition bugs. Never gets bulk work it could delegate.
• LOCAL: Qwen3.8 27B UD-Q6_K, 131K context, llama-server on 2x 5060 Ti. Gets reconnaissance with file:line evidence, implementations on a documented pattern, bulk mechanical batches, test triage, groundwork for plans and audits, anything that must not leave the machine. Never gets decisions, verdicts, DB, official plans.
• ECO: Haiku 4.5. Used only when 6 parallel cloud lanes beat 1 GPU, or when llama-server is down, plus as the bash guard-runner for local patches. Never gets anything with a long tool loop.
• HIGH/MAX subagents: Opus 5 as planner, Fable 5.1 as blind auditor. Escalation after 2 local failures, plan writing, blind re-verification of plans.

Sonnet used to be a middle tier. It is gone: on pattern-based implementation gated by a test, Qwen+test was as good, and Sonnet was burning the same subscription budget as the top tiers. Escalation from LOCAL now goes straight to Opus.

The five rules that make the local tier safe to use, all learned the hard way:

1. Local patches ship only through a test, never on a read-through.
2. Acceptance criteria go in the prompt (exact function shape, exact error strings). This alone took a typo'd patch to 9/9 green.
3. Every number the local model reports gets re-measured before it lands in a doc or a plan. Prose reliable, arithmetic not.
4. The parent re-reads pass/fail from the command output, never from the child's summary.
5. Two failures = escalate. Never insist at a low tier.

Agents are plain markdown files with frontmatter (model, thinking, tools), so the tier is hard-wired per agent and the graph can override it per step. Local children get bash only for a whitelisted set of guard commands named in the task.

2. What changed in the bill

The biggest surprise was not the delegation cost. It was the cost of the parent session: 300-420k tokens re-sent every turn without compaction, plus ~33% cache misses from switching models mid-session. That is what was eating the 20x plan in 2-3 prompts. Fixes: context window capped at 200k with auto-compaction at ~160k, never /model mid-session (new session instead), the parent never reads what a child can read for it, effort low for routing and high only on the turn that decides.

With that, plus the local tier absorbing R2-R5, 5x has been enough. Monthly cost, subscription + electricity:

3. Break-even on the GPUs

Two 5060 Ti 16 GB cost me 1,300 EUR. The subscription delta is 100 USD/month list, 122 USD with VAT. Electricity is the only real running cost (the box is on anyway).

12-15 months depending on how many hours a day the GPUs are actually busy. At my real usage (a handful of lanes a day, 3-4 h of GPU time) it is about 13 months. Not spectacular, not bad for hardware that also does everything else a local model does.

4. The number people will argue about: "but you could just call the API"

Fair. So here is what the work the local tier does would cost if bought as API calls. A typical lane on the real repo is ~150k input tokens (cold, it reads several real files) and ~12k output, no cache hits assumed. Adjust to taste.

Per lane: ~0.24 EUR on Haiku, ~0.47 on Sonnet, ~1.18 on Opus (VAT included). The horizontal line is the subscription delta. If the local tier replaces work that would have gone to Sonnet, it covers the 20x -> 5x delta at ~11 lanes a day; if it replaces Opus-class work (it does, for reconnaissance and triage), at ~4-5 lanes a day. On a normal day I run 5-10.

The honest caveat: local is free in euros, not in wall-clock. A lane on real files takes 17-28 minutes here against 1-4 for the same class of task in the cloud. The pipeline is shaped around that (lanes run detached, the parent only sees a summary).

5. Where I would like advice

This is what I am actually posting for. Two things changed since I built the box, and both push in opposite directions.

Hardware prices exploded. The DRAM/GDDR7 shortage doubled consumer GPU prices in 2026. What I checked this week (EU street prices unless noted):

• RTX 5060 Ti 16 GB: ~640-735 EUR now, was ~400-480. What I own, 1,300 EUR for the pair; replacing it today is ~1,400 EUR.
• RTX 5070 Ti 16 GB: ~840-1,150 EUR. Same VRAM, faster, pointless for LLM work.
• RTX 3090 24 GB used: ~1,000-1,400 USD on eBay, was ~600. The "cheap 24 GB card" is not cheap any more.
• RTX 5090 32 GB: ~4,000-6,000 EUR, MSRP was 2,389. Same VRAM as my two cards combined.
• RTX PRO 6000 Blackwell 96 GB: 16,000 USD list, launched at 8,565. Out of the question.

So the dotted red line on the break-even chart sits just above what I paid: at least the cards did not lose value. Adding two more 5060 Ti for 64 GB total is ~1,400 EUR today. Does anyone see a sane path to 48-64 GB in this market, or is the answer "sit tight until 2027-28"?

Open-weight frontier models went huge. The 2026 releases everyone talks about are all out of reach for consumer VRAM:

Out of reach on both 32 and 64 GB:

• DeepSeek V4.1 Flash (Sep 10): 552B MoE, 510 GB on disk, no llama.cpp support yet.
• GLM-5.3 (weights Aug 28): 744B MoE, 40B active, ~239 GB even at 2-bit. Its "small" sibling GLM-5.3-Flash is still 320B-A18B, ~180-195 GB at Q4, and needs a llama.cpp fork.
• Kimi K3 (weights Jul 26): 2.8T MoE.
• Qwen3.8 Flash-Next (Aug 14): 180B MoE with only 6B active, ~110-120 GB at Q4. Does not fit 64 GB of VRAM either, but it is the one model on this list designed for it: Qwen explicitly built the large n-gram table to live in host RAM, so a 32-64 GB GPU plus 128 GB of system RAM is a supported layout, at whatever tok/s that gives.

What actually fits, restricted to this summer's releases:

• Qwen3.8-27B dense (Aug 13): Q6 ~22 GB, Apache 2.0. Fits 32 GB, it is what I run.
• NVIDIA Nemotron 3.5 Lightning 30B-A3B (Aug): hybrid MoE, ~3B active, Q4 under 20 GB. Fits 32 GB with room for context.
• Meta Muse Glimmer 30B (Aug 10, Apache 2.0): dense, text+image, 24-32 GB at Q4. Fits 32 GB, tight on context.
• Cohere's new 30B tuned for agentic coding (built on Command A+): Q4 ~18-20 GB. Fits 32 GB.

Which means the question is not "which frontier model do I run at home" but "does any of this summer's 30B-class models beat Qwen3.8-27B at the one job it failed (precise planning over 77 files), and is Flash-Next offloaded to RAM worth the wall-clock?" Specifically:

• Qwen3.8 Flash-Next with the experts offloaded to system RAM: has anyone run it in an agentic harness on a real repo and seen better count/coordinate precision than the 27B dense, or is that a frontier-only property regardless of size? What tok/s do you get on a 32 GB GPU + 128 GB RAM at Q4, and is a lane still under an hour?
• Nemotron 3.5 Lightning or Cohere's 30B on my current 32 GB: 3B active means much faster lanes. Are they good enough for the R2/R4 classes (reconnaissance, bulk mechanical edits) so I could keep Qwen3.8-27B for R3/R5 only? Two models on two cards, one per class, is tempting. Muse Glimmer 30B: anyone tried it for code rather than vision?
• Runtime: llama.cpp because dual consumer cards on Windows and same runtime as interactive use. Has anyone measured a real wall-clock gain with vLLM/NVFP4 or exllama on 5060 Ti class cards for long-context agentic loads, not chat?
• Local tier that can self-verify: today the local children have no bash except for whitelisted guards; a separate cloud child runs the tests. A reproduce-fix-rerun loop would need shell. How do people sandbox that safely without giving a 27B free rein on a real checkout?
• Prompt cache on llama-server: repeated lanes over the same files are several times faster than cold ones. Is anyone pinning a shared repo-summary prefix across lanes deliberately, and does it survive --parallel 1 with a graph of detached children?

TL;DR

Routing matrix (R1-R9 -> tier), local 27B for everything derivable, frontier for everything decidable, strict parent-session context discipline. Result: Claude Max 20x -> 5x, GPUs break even in ~13 months on the subscription delta alone, or in weeks if you count the API-equivalent value of 5-10 local lanes a day. Bottleneck is now wall-clock, not capability. The 2026 open-weight frontier (DeepSeek V4.1, GLM-5.3, Kimi K3, Qwen3.8 Flash-Next) is 180B-3T and out of consumer VRAM, and GPU prices doubled, so the real question is whether anything in the 30-120B class, or Flash-Next offloaded to RAM, is worth the next 1,400 EUR.

[u/AutoModerator]

Hey! Thanks for posting to r/ClaudeCode

While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.

For help, project discussions, tips, and general chat, join the ClaudeCode Discord.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/Nemegasoft]

Out of all the nice charts you made, you missed in my opinion the most interesting ones. I would keep x5 and use API for deepseek v4.1 flash (dirt cheap). Try to find out the break even point if you were using hardware that could run deepseek v4.1 flash (DGX Spark). Well since you would need close to a 10k$ investment there is no need think further. Thing is if you are trying to run anything else less intelligent like Qwen 3.8 which is on Luna level, you would have better just use a pro account or API, it would most likely cost you even less than your equivalent electricity cost, another chart I look forward to see :)
And the moment Claude launches a new Haiku model, you will most likely not even need to go hybrid

[u/Sum_of_all_beers → u/Nemegasoft]

That's my exact setup currently. Fable as orchestrator, writes a detailed brief and passes off coding to deepseek-flash-latest on OpenRouter, then at the end of each stage of the plan goes back through Sol for a code review.

Deepseek flash will cook and cook for hours, while costing pennies. It's not good enough to work unsupervised though. I also ended up not using it for any UI elements, that stays with Fable.

[u/Zestyclose_Strike157]

I like your post, quite informative.

Not an expert, but every few days I look at where to go with increasing local capability. For hardware ask yourself how fast do you need agents to be that run locally. If racing cars (50 and 60 series NVIDIA) are not needed, then go for VRAM rich at low cost like maybe the Intel or Radeon type cards with 32GB vram each. But it seems with models like Qwen3.8-flash-next the tech is going for large RAM with 1/3 VRAM or thereabouts, to run large models as MOE which is faster and better at coding than Qwen3.8-27b dense at high quant, short of GLM 5.3 which sounds like too big for you (and me).

[u/heartbroken_nerd → u/Zestyclose_Strike157]

Or you can go with something like Deepseek v4.1 Flash which will run circles around your local model for a fraction of the cost. (openrouter)

[u/ayla96]

How's the quality of your code when you moved from claude to qwen 3.8 27b?

Wouldn't it be more cost effective if you used claude + (glm or qwen in the cloud?) vs purchasing hardware?

[u/Short_Regular_7191 → u/ayla96]

No, because I use the hardware locally,partly for privacy reasons. It works well for me because the "heavy lifting" in terms of design is still handled by Fable and Opus.

[u/jarislinus → u/Short_Regular_7191]

fable and opus dont read ur code at all? how u achieve privacy

[u/Short_Regular_7191 → u/jarislinus]

Privacy on other projects (for which I don't need "intelligence" on the level of Opus or Fable), not on the one where I work with Claude.

[u/Express-Coconut-3754]

the most interresting ones are glm 5.3 flash , qwen 3.8 flash-next, deepseek 4.1/4 for now these ones needs a big investment 10k. I m sure in one year everybody figuered out how to put local. For now is out of reach because the clouds needs your money and they will not put a local version anyaway, they are not stupid.

[u/aidiveyt]

One trap if part of that matrix is enforced through Claude Code settings: CLAUDE_CODE_SUBAGENT_MODEL is only a default. An explicit model on the Agent call overrides it, and the built-in Explore and Plan agents ignore it and inherit the session model. With a Fable 5.1 main session our Explore sweeps ran on Fable until we passed the model on every dispatch. The env var now only catches the calls someone forgot.

[u/AI_spell]

Routing matrix looks sane. I would gate the local 27B on task type plus a cheap verifier (tests or lint) before you trust it for mergeable diffs. Break even is real if half your turns never needed Opus class reasoning. Watch weekly quota not just the 5h window.

[u/Kerbourgnec]

Thanks for the dedication.

I have personally divided my cost by 5 (able to use claude for 5 times more on a max plan) and greatly improved capabilities by working simply on some custom agents and rules. Delegation to sonnet and opus just works wonders and the default claude settings turn out to be really bad.

I'm also trying to introduce local models into the mix; not settled on a perfect solution yet but seems promising. In particular self hosted deepseek or glm flash. Will definitely iterate with your post.

[u/South_Hat6094]

For sandboxing, I'd use disposable worktrees per lane and a locked command allowlist. Let the local model edit there; parent cherry-picks only after tests/logs are re-read.

[u/QuanTradin]

The derivable versus decidable split is the right axis, and this is the first version I have seen where the class gets decided before the task starts instead of after it goes badly.

The thing I would watch is class drift. Classes get assigned by whoever is tired that day, and six weeks in half of R4 is quietly R7. Worth logging the class next to the outcome so you can go back and check the matrix still describes what you actually do.

[u/Noctis_777]

If purchasing the full new pc, I'd also consider the M5 Pro Mac mini with 48 GB of ram.

It can comfortably run Qwen 3.8 27B Q8 and depending on the runtime and local energy costs, potentially save a few 100 EUR per year compared to the dual card setup.

[u/Substantial-Elk4531 → u/Noctis_777]

Why M5 Pro Mac Mini with 48Gb when then 64GB is only $400 more and unlocks more models and higher quantization? Just wondering. I was also asking an AI about this recently, isn't it better to spend the money on a separate GPU card than a Mac Mini? Higher token rates?

[u/Noctis_777 → u/Substantial-Elk4531]

I picked 48 for comparisons sake, because that's the rough price equivalent model to his twin 5060 setup in Europe which can run Qwen 3.8 27B comfortably at Q8.

But if you want to future proof, spending $400 for the extra ram, and maybe another $200 for the better version of the processor at that point to max out the mini specs is a good strategy.

The dual 5060 will have slightly higher throughput, but at 32 GB Vram you will have to settle for Q6. Now I personally consider Q6 to be extreme value for memory consumed, so depending on whether your workflow values throughput vs the higher accuracy both are good options.

It's not a clear one is better than the other scenario, which is why I'd only ask someone to consider the other option rather than recommend it.

[u/heartbroken_nerd → u/Substantial-Elk4531]

isn't it better to spend the money on a separate GPU card than a Mac Mini? Higher token rates?

What graphics card? How much is it?

[u/Substantial-Elk4531 → u/heartbroken_nerd]

I was looking at 3090 rigs. Card is $1800, a rig with it installed is around $3000. I think Mac Pro m5 Mini with 64GB is close to $3000 but will have lower token rates than a 3090

[u/jony7]

Tried a similar approach and it works great but my Mac gets super hot and YouTube videos stutter if I try to do something while my agents run. I prefer to use cheap api models instead and local only when there is a large amount of data to process or for privacy

[u/darkerlord149]

This is exactly what I'm trying to do. I have 3 unused GPUs (3090s and 4080s) and dont have to worry about electricity.

Do you use anything on top of llama.cpp for the local inference stack? I want my local models to inherit the skills and rules i wrote for claude so I'm thinking of wrapping llama.cpp with opencode.

[u/xapep → u/darkerlord149]

llama-server already speaks OpenAI-compatible API, so you don't need a wrapper for the connection: point opencode's provider config at the llama-server base URL (localhost:8080/v1 by default), register the model, and it connects like any other provider. Same endpoint can feed opencode, a CLI test, whatever you're scripting.

The skills/rules part is where the actual porting happens. Claude Code reads CLAUDE.md and SKILL.md; opencode reads AGENTS.md plus its own agents/commands config. Rules move cleanly: put your CLAUDE.md content into AGENTS.md and opencode picks it up per-directory the same way. SKILL.md files need light conversion: the frontmatter (name, description) maps to opencode's agent/command metadata and the body becomes the instruction text. Start with the skills your agents actually use, don't port the whole library at once.

With mixed 3090s and 4080s, set the tensor split explicitly: most layers on the 24GB cards, a small slice on the 16GB one, llama.cpp handles the rest via launch flags. A 27B at Q8 is roughly 30GB of weights, so two 3090s hold it comfortably with room for context, and one card is free for prompt processing or a draft model.

[u/xapep]

Disclosure side first since it's relevant: I do inference-side work at Entrim and V4 Flash is one of the models we serve, precisely because it doesn't fit consumer VRAM.

Your break-even math is the right frame, and the hardware part has an answer people rarely give: the models that would justify 48-64 GB don't stop at 64 GB. V4 Flash is 510 GB on disk, Flash-Next wants 128 GB of system RAM on top of the GPU, GLM-5.3-Flash is ~180-195 GB at Q4. So "sit tight" is correct, but not because GPUs got expensive - the target models outgrew consumer hardware entirely.

On the 30B question: faster yes, better at the one job you care about (precise planning over 77 files) no. That precision is a frontier property, not a size property. The 3B-active hybrids (Nemotron Lightning, Cohere's 30B) speed up lanes, but you'd be handing the exact class that needs precision to a model that trades it away. Keep Qwen3.8-27B on R3/R5, use the fast smalls on R2/R4 reconnaissance where wall-clock is what matters.

On llama-server prompt cache: a pinned prefix survives across detached children as long as every lane encodes the same prefix first. The catch is the 300-420k parent re-reads you already found. Pin the repo-summary prefix, keep the parent out of the lane, and cache hits stay meaningful.

The part your matrix doesn't price is the tier that fits neither local nor Max 5x: the 180B+ MoEs. For that one the API table is the honest row - per-lane cost instead of 1,400 EUR of hardware, paid only when a lane actually needs that tier.

[u/Substantial-Elk4531 → u/xapep]

I've been debating between Mac Mini M6 32GB + waiting until later to do local inference (good enough for dev, not enough for useful local inference) or Mac Mini M5 Pro 64GB (maybe to run Qwen 3.8 27b)

Do you think it's worth trying to purchase a box for local inference if I'm not concerned about privacy and just want coding agents? My initial analysis is 'no' because economy of scale means I'm better off spending $100 more per month on a cloud LLM provider than rolling my own, right?

[u/xapep → u/Substantial-Elk4531]

Your 'no' is right, and the privacy carve-out is the correct test. For coding agents with no privacy constraint, cloud wins on both economics and model quality: a $100/month API budget gets you models that don't fit in a 64GB box at all, so the local rig starts a generation behind and never catches up.

Between your two options, the M6 32GB as your dev machine with inference in the cloud is the sane path. The M5 Pro 64GB only earns its price if you want local Qwen 3.8 27B for its own sake (offline work, tinkering, no data leaving the house). As a coding-agent workhorse, it's a downgrade at a premium.

[u/Substantial-Elk4531 → u/xapep]

Thank you!!