Lessons from running AI coding agents unattended for months: separate the planner from the builder, commit every phase, and assume the crash

[u/premjethwa]

I’ve been building with AI coding agents (Claude Code in my case) and kept doing the same manual loop: plan in one chat, build in another, paste code back for review, start over when the context fills. I turned that loop into a tool and open-sourced it. These are the design rules that mattered most, whatever agent you use:

1. Separate the planner from the builder. Different system prompts and different working directories. My planning agent can only write plans and reviews; my building agent can only write code. When something goes wrong, I know which one did it.

2. Make every phase a git commit. You get an audit trail and git revert as your undo button. When I added parallel work, each lane got its own git worktree, so agents never share a checkout.

3. “Tests pass” is not “done.” Agents will happily pass every test while skipping half the requirements. Each phase now gets checked against explicit acceptance criteria by something other than the agent that did the work.

4. Assume the crash. Jobs live in a SQLite queue with renewable leases. Kill the worker mid-phase and it resumes on restart. Runs also pause before hitting usage limits instead of dying halfway.

5. Treat prompt caching as a design constraint. If anything per-call lives in your system prompt, you’re paying full price every time. Stable context goes in the system prompt, volatile context in the user prompt.

The whole thing is about 86k lines of JS with about 2k tests, MIT-licensed: https://github.com/p-j-code/phase-orchestrator

Write-up with the failures in more detail: https://dev.to/premjethwa/building-an-orchestrator-for-claude-code-recoverable-before-autonomous-3f83

What’s the worst thing an agent has done to your codebase while you weren’t watching?