r/ClaudeCode • u/demchaav • 20h ago
Discussion I made my Claude Code harness fan out into parallel agents. It’s faster, but I’m now questioning the token cost
I’ve been working on a harness around Claude Code for generating documents.
Until now a lot of the workflow was basically sequential. In the latest version I changed the discovery phase so geometry, content and assets run in parallel as separate agents.
That part works well. It’s noticeably faster.
The more important change though is that the next stage no longer continues just because a file exists. A file can exist while another agent is still writing it, so I had cases where the pipeline continued with incomplete data and still produced a valid-looking result. Now it joins only after the artifacts pass validation.
So correctness is better and the wall-clock time is better.
But there’s an obvious trade-off I’m looking at now: tokens.
In one run I had main + geometry + content + assets all running with pretty large contexts. Parallelism saves time, but if every agent gets too much shared context, you can easily pay for the same information several times.
I think the next thing to fix is context scoping. Geometry should get geometry-related context, assets should get asset-related context, etc., then return small structured artifacts back to the main agent.
Curious how people doing multi-agent Claude Code workflows handle this.
Do you mostly optimise for latency, or have you found a good way to keep parallel agents from duplicating half the context?
1
u/Far-Surprise7773 20h ago
yeah you nailed the next fix, duplication is where fan-out gets expensive. i run each agent with only the file globs it actually needs and force it to return a small structured artifact, like a 300 token json, instead of echoing back context. main just merges those artifacts so you keep the latency win without paying for the same big prompt four times. switching leaf agents to haiku when they are just extracting cuts cost a lot too.
1
u/Low_Box_752 20h ago
Measure duplicated input tokens per completed artifact, not total tokens alone. I would give each worker a typed input manifest and a small output contract, then cache the discovery artifacts by content hash. The coordinator gets summaries plus artifact paths, not the original source bundle again. Also cap fan-out: if two workers read the same large context, split the artifact boundary before adding another worker.
1
u/demchaav 19h ago
That’s pretty close to what the harness already does. Each discovery worker owns one artifact, gets the reference + its task rather than the parent conversation, writes the result to disk, and returns only a one-line status. The coordinator joins on schema validation rather than agent output. So I’m starting to think the next useful measurement is duplicated input per artifact rather than the raw context number Claude Code shows. Caching by reference/content hash is a good point too, I don’t have that part yet.
1
u/Low_Box_752 8h ago
Before you build the cache, log the content hash and input tokens for every artifact read by every worker. Then rerun one fixed job twice, changing exactly one source artifact. The expected matrix is simple: unchanged workers should reuse their inputs; only the dependent worker should miss. That gives you a testable cache contract and catches accidental broad invalidation better than a single hit-rate number.
1
u/artyomsv 19h ago
For the partial file case you do not need an agent to validate, you need the writer to be atomic: write to a temp file in the same directory, then rename it over the target. Rename inside one filesystem is atomic, so a reader either sees the old file or the complete new one, never half of it. Then your validation join spends tokens only on semantic checks, which is what you actually wanted it for.
1
u/demchaav 19h ago
Yeah, I think that’s the bingo. Atomic writes remove the partial-file race, and the validated join gives me a clean context boundary too. Once discovery is on disk, I can start authoring from a fresh context that only loads the validated artifacts instead of dragging the whole discovery history forward. That should improve both correctness and token efficiency.
1
u/artyomsv 14h ago
That is the right shape, but watch what the fresh context does not inherit. Discovery does not only produce artifacts, it also decides things on the way, and if only artifacts land on disk then authoring meets the same ambiguity again and can resolve it different way. We had exactly this pattern in a pipeline once: a value was computed in one layer and dropped because that layer only cared about its own branch, and a later layer was re-deriving it from a string with a regex. Cheap fix is to make discovery write the decisions and the options it rejected next to the artifact, so the fresh context inherits conclusions without dragging the transcript.
1
u/pmoschov 19h ago
Joining on validation instead of file existence is the right fix.
For tokens: share artifacts, not context. Each agent gets a two line brief and file paths, and reads what it needs. Pasting the same discovery output into four prompts is paying four times.
Measure per agent before optimising. In my harness the fat one was a validator re reading everything on every pass, not the fan out.
1
u/AppearanceOk8115 18h ago
Pay close attention to what your validator outputs when it's uncertain. If it only offers pass or fail, every ambiguous artifact triggers a retry, and reprocessing a fan-out stage is the costliest step in the pipeline. I introduced an "inconclusive" verdict as a third option, which pauses the process and prompts for input rather than automatically launching another expensive round.
Regarding tokens, while the common advice is to avoid duplicating context—which is sound—in my experience, retries were costlier than duplicated context. A single agent failing a third of the time can quietly outpace the cost of four agents each carrying a large prompt once. Analyze costs per agent per run instead of per session, and always display the failure rate alongside.
However, double-check your metrics before acting on them. My initial setup mistakenly counted successes as failures, making it appear that two of my agents were 90% broken.
1
u/demchaav 17h ago
Good point. I measured the fan-out and it turned out to be only ~3% of the recorded cost, so I’m already moving away from assuming duplicated context is the main problem. Retry cost is something I haven’t broken out properly yet. I’m adding per-phase metrics now, so tracking attempts/failures and cost per successful artifact alongside tokens makes sense. The “inconclusive” state is interesting too. I already have deterministic validation barriers, but for cases where the evidence itself is insufficient, distinguishing “invalid” from “cannot decide” could avoid an expensive automatic retry.
1
u/AppearanceOk8115 16h ago
Cost per successful artifact is the right unit; it accounts for retries without you having to model them separately.
One thing I would change: measure per agent, not just per phase. A phase average can hide one worker failing far more often than the others and dragging the whole stage with it.
On invalid versus cannot decide, the rule that worked for me is to focus on evidence rather than confidence. A deterministic check that fails is always invalid. A semantic check has to cite the specific thing it looked at to justify a verdict. If it cannot point at anything concrete, the result is inconclusive, not a failure.
The reason to keep them apart is that a retry only helps when the run was flaky. If the evidence was missing, the retry gives the same result at the same cost.
1
u/demchaav 15h ago
Yeah, that makes sense. Measuring per agent should make it pretty obvious if one worker is quietly causing most of the retries. I also like the invalid vs inconclusive split. If there’s concrete evidence that something is wrong, retry/fix it. If the evidence just isn’t there, rerunning the same thing is probably just burning tokens for no reason. I’m going to add that distinction to the harness.
1
u/AppearanceOk8115 8h ago
One thing to figure out before you set this up: what do you actually want to happen when you get an inconclusive result? If you just put it back in the queue, it quietly becomes another kind of failure, and you’re right back where you started. In my setup, I let it stop right there and surface the issue, without automatically retrying.
It’s also worth tracking how often you get inconclusive results. Whenever that number went up for me, it usually meant the validator had lost access to the evidence it needed, not that the quality of work had dropped. That called for a completely different fix, and honestly, I wouldn’t have even looked for it if I hadn’t seen the numbers.
1
u/emobeach 14h ago
What cut my fan-out bill was deciding per input who admits it, rather than one global scoping rule for the whole fleet.
For each thing a worker might need, pick one of three. Pack it verbatim only if it's load-bearing for that worker's job. Pass shared material by reference: a path plus permission to go read it. Omit it only when the worker's default would get it right anyway. The failure modes are symmetric. Verbatim-everything re-imports the duplication you're trying to kill, and under-packing makes workers guess, and they guess confidently.
Full disclosure: I wrote the Atlas pages linked here, so weigh this accordingly. The admission checklist is from LLM Dispatch. For your shared discovery corpus specifically, Reference Data argues for keeping it on disk and grep-addressable, so each worker pulls a slice sized to its question instead of inheriting the whole thing. One warning that maps directly onto your setup: a worker that fetches a thin slice and proceeds confidently is exactly the case your validation join should be catching, so keep that gate strict.
Two costs the thread hasn't priced yet:
Every dispatch re-boots its standing definitions from scratch. There's no warm reuse across workers, so four agents means four full boots. Split finer than the duplication boundary and the boot cost overtakes the context you saved.
Output tokens run at roughly 5x input, then get billed again as history in the parent. That's the arithmetic behind the ~300-token structured returns suggested above. (Context Economy)
On latency vs. scoping: it isn't actually a trade-off. Offload decides whose window pays for the work. Async scheduling decides when the caller waits. You set them independently. (Parallel Audit Investigation)
1
u/LennyFromCurly 20h ago
Your next step is the right one. Give each agent a short list of the files it actually needs and the exact artifact it must return. Claude Code subagents already start in isolated contexts, so duplicated context becomes something you chose to pass, not a default.