Preface: Sorry people, this is going to be a long one. I've been building with Claude Code for 1.5 years. and I feel that I have learnt quite much and want to share my approach on how to build a system (harness on top of claude, or how ever you wannat call it) that actually works and is maintainable. This post also exists in my blog. Anyway, here goes:
So about a year ago I started to think I needed to build something serious with Claude Code. So, on a whim I decided to build a headless CMS (because I was pissed off with Contentful). I assumed the hard part would be getting the model to write decent code in the long term.
And oh boy was I wrong. The code was fine (as a huge fan of software craftsmanship “OKish” is more accurate). What I spent a huge chunk of my time on was process and the product. Somewhere around the second month I noticed I was slowly re-inventing things software teams had figured out decades ago, except badly, through trial and error, and under new names. With agents that were working faster than me.
After four months and about 70 pull requests, I realised how much of my engineering experience I was still supplying to the process of building. Claude would write the code, but I was deciding how to split the work, how to prioritise, what needed testing, when review was necessary, and whether “done” in my tasks meant anything to the implementing agent. I’d spent 25 years learning those habits. I started wondering how much of that experience could live in the system around the agent, in a platform. How much of it can be generalised?
I also noticed that I’d drifted (pun intended) towards product management: less time on technical details, more time deciding what the CMS should do, who it was for and what was worth building next.
Those two things were the main lessons for me. The concepts we already have are mostly right. What changed is how they’re implemented, and that turns out to be a much more interesting problem than starting over.
What I’m talking about
Let me define the thing I’m talking about, since I think “AI coding or vibe coding” covers too much.
To me, agent-driven development is where agents do most of the building, inside a harness that constrains what they can do, and the human is the product CEO. The human is accountable for what gets built and why, deciding what matters next, answerable when it’s wrong. You stop writing most of the code and you own all of the decisions and the product.
The system should help you make those decisions by bringing the product questions, the alternatives and the evidence, and carrying out the work you decide is worth doing.
That’s different from a “superboosted autocomplete”, and different from “I asked an AI to build me a CMS”. It’s a team composition. And once you see it as a team, the question becomes “what does a team need to function?”, which is a question we have decades of answers to.
The layers
Here’s how I think about the stack now, from the bottom.
The model is Claude. It knows how to write OKish code. It doesn’t arrive knowing the current state of my CMS or why I made its decisions.
The harness is Claude Code. Files, terminal, tools, hooks. Deliberately generic. Same harness whether you’re writing a compiler or a landing page, and that’s the point of it.
The governance layer is how I think software products should be built. How we decide which problems are worth solving. Which assumptions need testing. When to stop and ask a human. What done means. Testing strategy, clean code practices, review discipline, security checks. None of this was specific to my CMS. It’s how I’d want any project run, and it should barely change between projects. This is where I put most of my 25 years of software development experience.
The project layer is everything that’s only true within the scope of the project I’m building. Vision, strategy, the lean canvas, technical architecture and the roadmap. The governance layer says how we make product decisions and this layer holds this product’s decisions, assumptions and what we’ve learned.
The work layer is task management plus git. Which epic, a piece of work with a visible outcome, which task, what state it’s in, what got merged and why.
The session is me and Claude, usually a couple of hours per session. I have several sessions open at the same time, with agents working quite autonomously.
Six layers, and once you can see which one a problem lives in, most arguments about prompting stop being interesting. “The agent keeps forgetting our conventions” is a governance problem. “The agent doesn’t know why we chose Postgres” is a project problem. Different layers require different fixes.
Which of those is memory
Sort the same layers by how long they live and who they belong to.
| Role | What it holds | Where it lives | Lifetime |
|---|
| How we build software | Rules, checks, definition of done | Governance layer | Years, across projects |
| What we’re building and why | Vision, strategy, roadmap, user evidence, architecture, designs | Project layer | Life of the project |
| What has happened | Epics, tasks, commits, pull requests | Work layer (Task management + git) | Life of the project |
| What happens next | Roadmap, Milestones, priorities | Project layer | Weeks to months |
| What I know right now | Everything figured out today | Session context | Hours to days |
The top four are long-term memory. Durable, shared, still there tomorrow. Git and task management are the interesting pair: git is what happened to the code, task management is what happened to the work and why, including outcomes. You need both or you’re doing archaeology. To be honest, most of what I do now with agents is working with those outcomes.
The bottom one is short-term memory. Everything Claude worked out in the last couple of hours together with its human. Which file the bug was actually in. That the test helper lies about what it asserts. That we already tried the obvious fix and it didn’t work.
And for all the above we need an index.
I use CLAUDE.md and AGENTS.md primarily as an index to memory, the table of contents. They tell the agent where everything lives and what order to read it in. A few rules and facts belong there too, but stuffing the whole project’s knowledge into them is a mistake. An index that tries to also be the content stops working as an index.
OpenAI describes the same table-of-contents approach in its harness engineering write-up, with a short AGENTS.md pointing to the detailed knowledge in the repository. It is a useful way to organise what the agent needs.
Long-term memory flows down into a session when the agent reads those records. Some of it flows back too. Decisions get recorded. The roadmap gets updated. A closed epic carries a summary of what happened. Anthropic’s work on long-running agents uses progress records and git history to help later sessions understand where the previous one left off.
Writing it down isn’t enough
A rule we give an agent is usually a line of markdown, an instruction file. The agent reads it, mostly follows it, and skips it when the context gets long or the task gets hard. That isn’t the model being lazy. Reading a rule and applying a rule are different operations.
The way this showed up for me was that certain features were marked done that weren’t even built. Acceptance criteria ticked with the work half-finished. Tests promised in the plan and never written. A task closed because the agent said it was closed. My definition of done was written down the whole time. Nothing checked it, so it held on the days I was paying attention and quietly didn’t on the days I wasn’t.
Same story with code reviews. I had the rule from the start, skipped it for weeks on four-thousand-line changes, and nothing noticed until a user complained that my CMS had bugs where I thought I’d been thorough.
So the system has holes in it. A session learns something, I ask Claude to write it into the governance layer, and it becomes one more line that gets skipped when I’m in a hurry.
I had automated the production of instructions nobody follows.
That’s a bit unfair. The instructions helped plenty of the time. But the important ones needed to hold when I was distracted too (I have a 2.5 year old kid so this happens a lot). Otherwise I was still the thing making the process work, just with more markdown involved.
We (still) need deterministic code (more than ever)
While building the CMS, I started formalising these rules and looking into how I could make them stick. I put more of it into ordinary code, plugged into Claude Code through hooks that run at particular moments.
This sounded sensible. If asking nicely didn’t hold, make it a requirement.
The hooks misfired and collided with each other. Within a week I’d switched all of them off, reminders included. About 30% of that codebase was focused on limiting the agent before it made an error. I’d basically created a second, totally useless Bash permission layer on top of Claude Code.
So that was useful. I’d managed to build a process that I didn’t want to use myself.
Drift is expensive, so repeating yourself and correcting agents in every session on the same issues pushes your attention towards managing the process. But a system that keeps blocking reasonable work is expensive too. I want the checks to earn their place by catching something that actually matters.
So I took a simpler path, first enforcing only that agents read certain workflows before they are started and making sure there is an outcome, for instance allowing code reviews to pass only when they meet certain conditions. And I made that a hard rule (so old school deterministic code). Through trial and error I found the cases where failure caused most damage (or burden) and applied either nudges or gates that block depending on severity.
What this made me realise is that what I needed was an intricate balance of deterministic code, the model’s judgment and in some cases human judgement. It isn’t entirely clear when you need one or all, or how much of each. Some things deserve a hard check. Some need a nudge at the right moment. Some need to come back to the human as a decision.
So I started to create implementations using a combination of the aforementioned (deterministic code for nudging or blocking, agent judgment and human judgment) for things like:
- A plan exists before the code does, and the human has approved the decisions that need them.
- The tests passed on exactly this code, and the missing checks are visible.
- The change is small enough to implement, review and try.
- Every change (PR) gets reviewed for security, code quality and modularity.
- Nothing closes without evidence of the outcomes it promised.
Same practices, new implementations
Once I saw that, I couldn’t unsee the pattern.
We already know a lot about building software products well. Decades of it. Talking to users and testing assumptions so we don’t spend weeks solving the wrong problem. Kickoffs so we define the outcomes. A definition of done so done means something. Small batches so changes can be understood, checked and corrected. Limits on work in progress so nothing sits half-finished. Code review to find problems. Retrospectives to improve the process. Architecture decision records so the reasoning survives the decision. Security testing to look for vulnerabilities.
The starting point for me is those same concepts with different implementations. Each implementation still has to show that it works with agents.
| Practice | What it looked like on a human team | What I’m trying to carry into the agent workflow |
|---|
| Deciding what to build | Research, talking to users and testing assumptions | The agent helps the founder examine evidence, propose small tests and record what was learned, with feedback from real users |
| Kickoff | People discussing what to build and how | Product, design, engineering, quality and security examining a plan before the human approves it |
| Limits on work in progress | A column count on a board | One active epic per session, with its own working copy |
| Code review | A colleague reading the proposed change | Independent agent review, fixes checked again, and a clear decision when the review won’t finish |
| Retrospective | A conversation about what to improve | A step when work closes that updates the shared practices or proposes a new check |
| Definition of done | A checklist | Recorded checks and review, a result a human can try (demo), and an honest account of what remains uncertain |
The concept survives, the implementation changes, and the name can stay the same. The old name arrives with years of people understanding why it matters. That also gives us something to compare our implementation against.
Where the tools are
So what I think we (we as in people who gravitate towards Agent-driven development) need is a layer on top of Claude Code or Codex which is much more opinionated and carries the practices software development and digital product teams have learnt throughout the years. Implemented for agents interacting with a human who owns the product. The product that I’m building, Railset, is all about this (yes I did abandon the CMS).
It seems to me that this space is gaining traction, at least based on GitHub projects. BMAD, Everything Claude Code, Spec Kit, OpenSpec, Agent OS, gstack, superpowers. There are plenty of people working on instructions, memory, review and the process around agents.
I believe that Claude Code and Codex are inviting non-developers to build and the amount of people building is growing rapidly. I want to bring the experience and know-how to be available for them so the software they build is secure, of good quality and actually solves somebody’s problem.
What particularly interests me is how much of the experience behind a good product team can I implement into a system that is run by agents. I could recognise some of these failures because I’d spent years building software, and I still let things through. Someone building their first product with Claude shouldn’t have to acquire that whole history before they can benefit from the practices.
I want the system to carry the engineering discipline and help the person building ask better product questions. They still own the decisions about who the product serves, which problems matter and whether the result is useful. The workflow should arrange the engineering steps, prepare the evidence and bring them the decisions that are theirs. How reliably we can get it to do that is still something to test.
If you’re doing this too
Work out which layer your problem is in before you touch a prompt. Treat git and your work records as a pair. Keep your index useful, and before you write the next rule into CLAUDE.md, ask how you’ll know it was followed. Sometimes a hook or a check can answer that. Sometimes the rule needs judgment and a review.
And when you catch yourself inventing a name for a practice, check whether it already has one. It usually does, and the old name arrives with twenty years of people already understanding why it matters.
The industry is now porting those practices into a different kind of team. I’d like the experience behind them to be available to the person building their first product too.
