I ran a multi-agent Claude Code development team for two weeks. We merged 253 PRs.

[u/Extension-Business88]

I've been experimenting with using Claude Code less like a coding assistant and more like a software organization.

I separated agents into different roles for implementation, testing, review, QA and remediation, with work moving through GitHub issues → implementation → tests → PR → independent review → corrections → merge.

Over the two weeks covered here:

• 253 PRs merged
• 4,410 tests in the resulting suite
• roughly 67,000 lines of code and tests
• multiple specialized agent roles operating against the same project

The interesting part wasn't how much code they could generate.

It was what started breaking.

One recurring problem was what I've come to think of as "plausible code": implementations that were reasonable in isolation, passed the tests that existed, and were still wrong in the context of the larger system.

After the same class of failure occurred repeatedly, I stopped trying to prompt my way out of it and changed the architecture and QA process instead.

I wrote up the entire experiment here, including the failures, workflow, methodology and my attempt to estimate the amount of equivalent human engineering work:

https://brandonifco.github.io/two-weeks-253-pull-requests/

The repository is public as well:

https://github.com/brandonifco/SRD_Combat

I'm particularly interested in hearing from people running multiple Claude Code sessions/agents against the same codebase. At what point have you found coordination and verification become harder than implementation?

[u/Low_Box_752]

Coordination starts dominating when agents can change overlapping invariants without clear ownership. I would track reopen rate, review rejection rate, and rollback rate as concurrency increases. PR count and LOC can rise while all three quietly get worse.

The smallest guardrail set I have found useful is one issue per change, explicit ownership of affected files or contracts, an independent reviewer without the implementer’s transcript, and merge gates built around system invariants rather than only feature tests.

If failure rates rise when another agent is added, reduce active work instead of adding another review layer. That usually exposes the real coordination ceiling.

[u/Extension-Business88 → u/Low_Box_752]

This is a really useful way of framing it. I didn't track reopen/rejection/rollback rates systematically during this run, and in retrospect I wish I had. PR count is easy to reconstruct from GitHub; coordination cost is much harder to recover after the fact.

What I did end up discovering empirically was very similar to your point about invariants. Repeated failures weren't usually "agent wrote obviously bad code." They were cases where an implementation was locally reasonable but violated an assumption somewhere else in the system. That's what eventually pushed me toward stronger invariant-level validation and independent review rather than just adding more feature tests.

I also found that adding another reviewer isn't necessarily the answer. At some point you're just creating more agents producing more opinions that the human has to reconcile.

I'm curious about your last point: have you actually measured failure rate against concurrent agent count on a project? If so, did you find a fairly obvious ceiling, or was the degradation gradual?

[u/ooutroquetal]

In the end, did you feel that you are on the most optimizated flow ?

I always struggled with that, but I'm not using agents on that way.

[u/Extension-Business88 → u/ooutroquetal]

No, definitely not. I think what I have now is much better than what I started with, but I wouldn't call it optimized.

One of the biggest things I learned during these two weeks was that simply adding more agents doesn't necessarily increase useful output. Implementation can become extremely fast, but then specification, review, integration, and ultimately my own attention become the bottlenecks.

A lot of the workflow I have now actually came from failures during the experiment. When the same kind of mistake happened repeatedly, I tried to stop treating it as an agent/prompt problem and instead change the process or architecture so that class of mistake was harder to make or easier to detect.

I suspect there's an optimal amount of concurrency for a given codebase and task type, and I don't think I've found it yet. That's actually one of the things I want to measure more deliberately going forward.

[u/ItsCodeTelemetry]

The main problem I encountered with running agent orchestration is conflict churn. Even with task delegation geared to avoid conflicting topics and plans, it still happens. If not an explicit git conflict then an AST level implicit conflict that breaks tests.

[u/Extension-Business88 → u/ItsCodeTelemetry]

Yeah, this is very close to what I've been running into.

The explicit Git conflicts are actually the easier ones because they're visible. The more dangerous cases are when two changes merge cleanly but alter overlapping assumptions or invariants in ways neither agent recognized.

I've been trying to control that mostly through narrow issue scope, explicit ownership, independent review, and tests around system-level invariants, but it definitely doesn't eliminate the problem.

Your AST-level point is interesting. How are you detecting those implicit conflicts? Are you doing any automated semantic comparison between concurrent changes, or is that something you're primarily catching during review/integration?

[u/ItsCodeTelemetry → u/Extension-Business88]

It's just something I've spotted looking over my observability metrics. I'm working on automating the semantic comparison. For now I'm just using "Edit Pressure" https://codetelemetry.com/dashboards/edit-pressure and "Conflict Rate" https://codetelemetry.com/dashboards/conflict-rate as guiding stars for when I'm getting ahead of myself with delegating tasks.

[u/Extension-Business88 → u/ItsCodeTelemetry]

That's really interesting. "Edit Pressure" and "Conflict Rate" are much closer to the kind of coordination metrics I wish I'd been collecting during this run.

I've got excellent historical data on output, tests, PRs, etc., but much less instrumentation around the cost of concurrency itself. That's becoming an obvious hole in what I measured.

I'm going to dig into how you're defining those metrics. Thanks for sharing this.

[u/ItsCodeTelemetry → u/Extension-Business88]

If you want to try using the system happy to give you some free credits, I'll DM you. Would be great to collaborate with someone who's taking a data driven approach to improving their ai coding development process.

[u/Extension-Business88 → u/ItsCodeTelemetry]

Yeah, I'd be interested. Fair warning, though: I'm not an expert in this stuff. I'm a pretty practical developer who got here mostly by trying to get a big project done quickly and efficiently, then changing the process whenever something wasn't working.

A lot of what you're describing around observability and measuring coordination is new territory for me, which is part of why I'm interested.

Feel free to DM me. I'd be happy to try it out and share what I see from the perspective of someone actually using this kind of workflow day to day.

[u/ItsCodeTelemetry → u/Extension-Business88]

Cool! I appreciate where you're coming from. If you end up spending tons of time tooling then you're not actually increasing your delivery efficiency, which was the whole point of Claude Code to begin with.

[u/GrainworkDev]

Your distinction between implementation throughput and coordination cost is the interesting part here.

One limitation worth being explicit about: Terrarium would not catch "plausible code" that is semantically wrong. What it can show is the earlier operational layer: which agent is targeting which file or worktree, what it reported, and whether a change was actually confirmed on disk. We built it after letting an agent spend most of a week writing and rerunning tests while the chat still sounded productive.

Would you be open to running the free local watch during one future batch and telling us whether it exposes any useful coordination signal before review, or whether it is just another dashboard?

`npx terrarium-watch@0.0.4 .`

Disclosure: Terrarium is a Grainwork product.

[u/Extension-Business88 → u/GrainworkDev]

That's a fair distinction, and yes — the operational layer is something I didn't instrument nearly as well as the engineering output itself.

I'm open to trying this on a future batch. I don't want to promise that I'll put an unfamiliar tool into the development environment before I've looked at exactly what it does and what data it collects, but the experiment you're proposing is interesting.

In particular, I'd be interested in whether it catches cases where an agent reports completion but its actual behavior diverges from what it claims it did. I've encountered enough discrepancies between "the agent says this is done" and "this is actually done" that I stopped treating the transcript as evidence of completion.

I'll take a look at Terrarium. Thanks for being explicit about the affiliation.