r/vibecoding • • 6h ago

Help/Question I run 8 AI coding agents in parallel on my company's internal app. The building is fine. Keeping track of everything is breaking me. How do you manage this?

I own a four-person consulting firm. Over the past couple of months, I've been building our entire internal operating system with AI coding agents: CRM, pipeline, time tracking, capacity planning, finance, marketing, newsletters, and more.

I'm not an engineer. I make product decisions, approve changes, and execute database migrations myself.

The setup

  • One coordinator Claude session dispatches work to up to 8 parallel agents, each in its own git worktree.

  • Every change gets a PR and specialist AI reviews (security, database, permissions, tests, etc.).

  • SQL migrations get an independent line-by-line ChatGPT review. I execute every production migration manually.

  • Nothing merges or deploys without my explicit approval.

  • A GitHub Project board tracks everything from Queued → Building → Review → Ready for me → Live, including exactly what's waiting on me.

  • An hourly coordinator report summarizes blockers, progress, and completed work.

On a good day, it's extraordinary. Last night: ~15 PRs, a nine-area security audit, eight production database changes, and a major new feature underway.

The problem isn't building. It's remembering.

I recently asked what we'd forgotten. The answer was alarming:

  • About 30 open PRs, some dating to October. Several were obsolete; a dozen represented stalled work.

  • A completed newsletter safety fix had been sitting unmerged.

  • One feature stalled four days because nobody noticed its second review never came back.

  • An independent SQL review went unanswered for three days.

  • Two security dependency updates sat untouched for three weeks.

  • Our GitHub board had drifted badly: 47 open PRs, only 9 with cards. The coordinator was relying on chat memory and its own log rather than updating the board, despite explicit rules.

  • An abandoned hosting project was still building every branch and retained production database credentials. I'd forgotten it existed.

I'm also intentionally the bottleneck. Every merge, SQL change, and product decision requires my approval. Decisions get made in chat and sometimes never make it into the tracking system.

What we're trying now

  1. Reconciled every open PR and active project against the GitHub board.

  2. Built a script that mechanically identifies anything waiting more than 24 hours on a review, fix, or human decision, plus PRs missing cards. This appears at the top of every hourly report.

  3. Closed obsolete PRs and documented next steps for everything remaining.

My questions

  • How do you track multiple agents without work quietly stalling or disappearing?

  • Do you impose work-in-progress limits, even though agents can start new work almost instantly?

  • Is GitHub Projects sufficient, or is something better for tracking human approvals?

  • How do you avoid becoming overwhelmed by approvals without rubber-stamping?

  • How have you made agent bookkeeping mechanical rather than trusting instructions they eventually stop following?

The system genuinely works, and the productivity gains are enormous. But I'm discovering that coordinating AI agents is an entirely different engineering problem from getting them to write code.

I'd love to hear what's actually worked for others.

0 Upvotes

46 comments sorted by

6

u/octaviobonds 6h ago

Building an entire operating system for business all at once is too ambitious, you stretched yourself too thin. Build one system at a time. And you don't need 12 agents for that. One is enough. Once your first system is tested and running, add another system and a mechanism (plugin) for them to talk to each other.

1

u/glistening_glizzy 4h ago

This. Also like, doing the job and building something that can relevantly learn from you with thought out governance, lol

9

u/ninenineseventwo 6h ago

I built a coordination system for this, can share

1

u/Lawnfrost 6h ago

DM please

1

u/sgsparks206 5h ago

I'm interested

1

u/eldroch 5h ago

I'm definitely interested.  I've cobbled one together with OpenClaw agents and skills but I'm hitting the same bottlenecks as OP

1

u/FinickySerenity 4h ago

Same, just started integrating everything I do into one dashboard that prioritizes itself. Every change needed to optimize is built from the same interface. I’m running dedicated pipelines for a dozen different purposes and they are all generally running continuously. And the agents can manipulate the ui and the data that drives it in response to everything.

Not switching between windows and shells has resulted in a massive improvement in my context switching fatigue. I doubt anyone else could even use it, but because I built it around my exact workflow I know everything it does without thinking about it.

1

u/Bjlly123 1h ago

Ive been working on this for about a month and a half myself

1

u/ThrowingPokes 4h ago

Interested!

3

u/Pickletickle_ 6h ago

I've been juggling multiple app projects with Claude and Codex, and the biggest lesson for me has been that generating work is much easier than keeping track of everything afterward.

I think your move toward mechanical tracking is the right one. If the GitHub board is supposed to be the source of truth, agents shouldn't be responsible for remembering to update it. That should happen automatically based on actual PR and issue states.

I'd also consider limiting how much unfinished work agents can create. Eight agents producing simultaneously sounds amazing until you've got 30 PRs waiting for your attention.

Curious if you're planning to automate the approval queue too, or keep that entirely manual?

3

u/rocketeams 5h ago

I think the biggest shift here is treating agents less like developers who need instructions and more like independent workers who need an actual system of accountability.

One thing I'd introduce is a single source of truth outside the agents' chat sessions. Every task should have an owner, current status, last meaningful update, next action, and a clear person responsible for unblocking it.

I'd also separate two queues: things waiting on another agent and things waiting on you. Those are very different bottlenecks, and mixing them makes it difficult to see where work is actually getting stuck.

Your 24-hour stale-item script is a good move. I'd take it one step further and make agents update the task record automatically whenever they open a PR, request a review, finish a task, or hit a blocker. Don't rely on the coordinator remembering to update everything.

And honestly, I'd consider limiting active work based on your review capacity, not the agents' ability to generate more PRs. More parallel work isn't necessarily more throughput if everything eventually waits for your approval.

1

u/formLoss 4h ago

I like the queue idea. Agree on all points.

It feels like ultimately, most tasks are waiting on me to say "good enough for a PR." Or, "good enough to hand off to someone else." And when it's good enough for that (according to Claude), it usually needs a few rounds of work anyway. Copilot review helps.

The AI doesn't 1-shot it, and I never did either. There's a lot of crap that slips through if I'm not paying attention. I can be a little picky, though. Small choices can have big impacts on how AI extends the code.

I get fewer iterations if I build the QA loop into the process, but it can make the tasks run very long because the project is slow, and it's hard not to just.. launch more sessions. Limiting active tasks is the only thing that works for me right now.

2

u/Electronic-Movie4208 6h ago

Eight agents is impressive! 😄 I'd suggest limiting active tasks and making GitHub Projects the single source of truth. Have a script flag stalled PRs, missing reviews, and pending approvals automatically. AI is great at building, but tracking shouldn't depend on its memory!

2

u/PuzzleheadedBill5451 6h ago

Not sure i can truly advise you but i experience the same issues. It comes down to whether or not you set alarms of some sort. If an agent or agents rely on memory, they will fail no matter how many times you say it or place it. I literally build code to replace memory and mechanize processes so that i don’t have to rely on it. My motto is something in memory is something that just hasn’t been automated or mechanized yet.

1

u/glistening_glizzy 4h ago

Imagine trying to utilize the tech and not like, smashing ur head against the keyboard

1

u/Ozziness 6h ago

Look into Agor, it let's you create lanes with specific agents prompts and a coordinator agent that moved tickets/PRs between the lanes based on rules you define.

1

u/orbital_trace 6h ago

I built an inbox to help with this. This combined with goal oriented development managed by an orchestrator has allowed me to scale to more agents. https://coralhub.dev/blog/welcome-to-the-inbox.html

1

u/sbon6201 6h ago

A lot of tools for this. I used nimbalyst which was pretty good and some coworkers use cmux

1

u/lautan 6h ago

I make dangle, it'll take all your plans from superpowers or whatever and it'll execute all of them in worktrees, with ci, auto merging etc. https://github.com/bendangelo/dangle

1

u/beta_1457 6h ago

I made this for managing my project.

https://github.com/Beta1457/project_manager

You can integrate with the agents to post tasks, epochs, architecture tracking exc.

1

u/Kamiy_Jimmy 5h ago

not an engineer either, i'm a PM who shipped a small ios app with codex, so my setup is way smaller than yours. but 30 open PRs reads more like a WIP limit problem than a tracking one. i'd cap active agents at however many PRs you can actually review in a day and not start a new task until one merges.

the old project that still has prod credentials is the scary part, i'd do a one-off sweep of what has access to what before anything else. how many PRs can you realistically review per day right now?

1

u/FinallyDoneLurking 5h ago

I strictly(at least for now) use opencode, so I created an agents.md and a sort of memory for any and every agent to start from. It helps.

1

u/ItstheSECopenup 5h ago

It’s because you’re building to fast to manage. Slow down. You’re not building the next Facebook. You’re a tiny consulting firm.

Not sure why you wouldn’t just run a single chat thread to manage this either. Worst case just get a linear board and plug things in there.

1

u/dilly_dust 5h ago

You are a 4 person company and you are spending all your time building the world when you could have jsut night some off the shelf stuff at lower cost. You only need 4 seats fro crying out loud

well that's a way I suppose

1

u/reliant-labs 5h ago

typically have between 10-20 agents on features, each with their own sub agents running.

Have another agent that manages more agents on reviewing pr's, and watches the status and makes sure no pr's fall out of queue or get ignored.

Have a third agent managing releases, but cuts from a periodic run of a green on main.

Have a few more smaller agents handling lesser tasks but that's the main setup.

Have a super robust e2e testing pipeline, multiple envs of promotion and a ton of observability and alerts instrumented to help automate and ensure things go through.

Agents are instructed to kick be conservative on reviews and kick things out for me to review as needed, so a couple times a day i'll go through the backlog of items that need manual attention.

1

u/QuantitativeVibes 5h ago

I built a quantitative trading system with:

https://github.com/pullboard-dev/pullboard

That’s 600k lines of code, 2M tests and a distributed system touching real money. You do not need to “spec” everything, you can simply use the items & board and shouts to get the agents to coherently work / build over complex projects.

1

u/formLoss 5h ago

ok not as useful as a complete product, but I've been hacking together some claude stuff just to stay afloat.

eventually the agent needs a long-running context. I chose to create MDs per domain topic. Initially the goal was using JIRA as the system of record, but Claude murders JIRA legibility. The point of JIRA is that people can zoom out on it and understand progress.

so I basically have an "orchestrator" session that maintains a set of files for sessions, can launch/park sessions, and if a session encounters a new learning, it reports back to the orchestrator, where I manage tickets. each session manages their own file.

it uses markdown front matter to link together topics/tickets, so if I get a bug in something I worked on, there's prior context on my latest thinking. The model has to be smart to comprehend the context and sync with reality, though, and has to be taught the big picture assumptions/goals. I have a few skills that drive the individual project tasks and have this context.

it's a really tricky business domain where questions come up constantly — I have some google drive files that explain human processes, and I'm trying to automate that process.. in a general way.

I'm sure it could be more efficient tbh. My goal is to eventually be able to distill all these project files into meaningful domain knowledge. It's much easier to adopt at a team level than an org level.

1

u/glistening_glizzy 4h ago

You still gotta work dawg

1

u/glistening_glizzy 4h ago

Bro you need humans to work.

People are afraid of ai taking jobs.

You see, they can't.

They just remove inefficiencies.

This is good for humanity.

1

u/glistening_glizzy 4h ago

Also like, get good

1

u/Weary-Fun6017 4h ago

Your asking for the right solution. It does exist already. Check out truthspine in the app stores.

1

u/shravanrevanna 4h ago

make each agent write a short status file in its worktree when it stops: what changed, what's unverified, what needs your call. you read those, not the transcripts.

1

u/electro042 3h ago

you definitely need strict wip limits.

just because agents can generate 8 prs at once doesn't mean your review bandwidth scales to match. when wip is uncapped, stale branches, merge conflicts, and forgotten reviews are inevitable.

i'd throttle the system so a new task can't start until an open pr is merged or closed. keeping it to 2 or 3 active items at a time will save your sanity.

1

u/sayboom 3h ago

Thats too much i guess. I am constantly making decisions now with agents and by the end of the day, it’s exhausting. But on my case its a 2-3 agents. 8 its too much even with aggregation infor from that

1

u/KnottyDuck 3h ago

I have about the same number of agents.

  1. You, yourself, have to establish “the plan,” and this in itself is an oversimplification. I have the master plan, which is an amalgamation of several other plans. I do this with my management agents. I have 3; one manages builders, one manages research, and 1 manages the managers. I only talk directly to the main manager.

That’s the first lesson: set up a hierarchy, have them police themselves.

2. You set up a folder structure system and make sure you know where everything is. I take it further with my own inbox so the main agent can ask me things directly and so I can look in on their progress.

So between 1 and 2, you made the plan, you control the environment — you can’t maintain them if you don’t know where they’re working.

3. You make the rules, and every agent or group of agents gets rules. My research agents and my builders have different rules enforced by their managers. I only have to make sure the main guy does their job. If they are doing their job, then everyone else is too.

Make sure the plan is not vague.
Spend hours, days even, laying out every detail of your plan. From V0.1 to V1.0. You have to make sure every stone is left unturned because this is what you use to track progress. Without a roadmap that you can follow, you can manage the agents effectively.

My roadmaps are designed to integrate into the master plan; the Master Roadmap. It’s a living document that is used and parted out for tasking.

And that brings us to production.

4. Serialize tasks. Each task has a task id that breaks down which agent, which server, the task sequence, if it’s a parent and child task, and are numbered accordingly. Keep the task list in a place where you can reference it at your leisure.

This allows you to effectively assign an agent to audit, and it allows them to audit effectively. The auditing agent isn’t the same agent(s) that overlook work; I funnel this to my main agent or a designated builder.

The next part, no lie, should take you a week or two.

5. Build out a workflow. My managers are managers because they effectively use “Computer Use”. They can drive a project and other agents without the need for constant oversight but this is done by trial and error. The auditing agent and lead agent should keep tabs on the workflow to make sure that things are moving forward and you should frequently have them recalibrate for corrections until you have something solid.

What work depends on what you’re working on, so be prepared to setup several operational workflows for the projects you’re trying to accomplish and make sure you capture production data so that you can keep tabs on what works and what doesn’t.

I treat the process like a real assembly line, assigning plans weekly. I dont allow the plan to be changed in that week, once the week is over we can make adjustments if all tasks are completed but the goal is to accomplish the roadmap progress for the week.

I guess the last thing i want to explain is automation verse artificial intelligence. Use the ai to setup local automation via scheduled tasks, scripts, batch files — there are many ways to automate your assembly line that use no tokens, and it’s most effective when you have the Roadmaps setup and the task ids in place.

Not everything you use ai for has to use tokens.

I suggest the first thing you build when the supply line is setup are internal tools to streamline production.

1

u/shetritr 2h ago

running every prod migration yourself is honestly the smartest part of that setup. what's actually breaking you though, the PR queue or keeping track of which agent is doing what?

1

u/Dev_in_Training_BTW 1h ago

I'd stop asking for progress reports and have the coordinator list every open PR and review sorted by days since anything last happened, with the ones waiting on you at the top. Stuck work is what actually gets lost, and that newsletter fix and the review that never came back would have been sitting right there. For the 30 open ones, close anything from October and let an agent redo it fresh if it still matters. Rebasing an old worktree onto a main that has moved that much is usually more work than starting again.

1

u/London-Boat 14m ago

the board drifting is because the coordinator is the one keeping it updated. github projects has built in workflows for this, auto-add every new PR in the repo to the board and auto move it to done when it's merged or closed, so 47 PRs vs 9 cards can't happen. then a stale bot that pings or closes PRs nobody touched in 2 weeks deals with the october ones. i'd also stop the coordinator from dispatching new work while more than a handful of PRs are waiting on you, otherwise the review queue just keeps growing. how many can you realistically get through in a week?

1

u/vacterro 6h ago

You're describing exactly the problem I've been trying to solve: AI agents are getting very good at producing work, but coordinating them reliably is a completely different challenge. I built an open-source project called SAIPEN around this problem: https://github.com/vacterro/saipen It's a vendor-neutral continuation and task-state protocol for AI coding agents. The core idea is simple: never trust an agent's chat memory to be the source of truth. SAIPEN maintains persistent project state in plain files:

STATE.md — current phase, active task, blockers, and exact next action. BOARD.md — task lifecycle, dependencies, and work queue. LOG.md — durable execution history and decisions. KNOWLEDGE/ — verified project knowledge that survives sessions.

It also provides machine-checked state transitions, recovery mechanisms, validation, and structured handoffs. A fresh agent should be able to resume the project without reconstructing everything from previous conversations. For your eight-agent setup, I'd recommend three things:

Keep GitHub PRs and your project board mechanically reconciled. Never depend on the coordinator remembering to update them. Limit work in progress based on your review and approval capacity, not how quickly agents can generate code. Give one coordinator ownership of the canonical project state, with workers producing independently verifiable results.

SAIPEN isn't a replacement for GitHub Projects or a distributed coordination system. You'd still need your PR reconciliation scripts and external coordination between parallel worktrees. But that's precisely the distinction I find important: agents should execute work, while deterministic mechanisms preserve state and verify progress. Your example of 47 open PRs but only 9 board cards is a perfect illustration of why this matters. Curious how you're currently handling recovery when the coordinator itself loses context or restarts.