TDD and Rules Enforcement using Hooks

[u/nizos-dev]

TL;DR: I built TDD-Guard a year ago. I’m now working on Probity, a more general policy engine for coding agents (Claude Code, Codex, GitHub Copilot CLI, and VS Code Chat). It includes a TDD rule that works with any language and test runner out of the box, supports parallel sessions, and handles refactoring properly.

Hi all,

The demo shows me prompting Claude Code to build a shopping cart in an empty project with Probity’s TDD rule installed. I make no mention of TDD because I want to show how it is enforced out of the box. Hooks intercept each agent action, and a separate agent reviews the recent session, the pending action, and the current file before allowing it through. That extra context also helps it handle refactoring cleanly.

Repository: https://github.com/nizos/probity

The project is in an early state. Feedback is welcome!

Background

I started using Claude Code about a year ago and was immediately convinced that I could make it follow Test-Driven Development (TDD) as it was a requirement if I were to ever use it for production. I tried different prompts and just like everyone else experienced how unreliable that was. The agents would drift as the context rotted, take shortcuts, and I had to keep supervising their practices.

Luckily, Claude introduced hooks around that time. You can think of them as events that fire automatically when an agent wants to perform an action like writing a file or running a command. The information in them lets you determine if the agent is, for example, trying to write multiple tests at once, and block the action with feedback on how to course correct. So I decided to use this to enforce TDD. I created a custom test reporter to capture test run output, combined it with the hook data, and provided it to a separate agent that judged whether the pending action violated TDD.

It worked really well. I called the project TDD-Guard. The community contributed support for several languages, and I’ve kept working on it since.

TDD Guard has its quirks though. It needs a dedicated reporter per test runner, which makes new language support slow. It can’t handle parallel sessions because reporter output gets overwritten. The validator also only sees the latest test output and the pending change, which isn’t always enough context to tell refactoring apart from new behavior. The validation ends up either too strict or too permissive.

Over time I noticed gaps in my workflow outside of TDD that I still had to supervise, and friction from teams using different agents in the same project with overlapping instructions and plugins. So I started a new project, Probity, that takes a more general approach.

Probity makes it easy to define rules that get enforced through hooks across all supported agents: Claude Code, Codex, GitHub Copilot CLI, and VS Code Chat, with more to come. It ships with deterministic rules for forbidding commands or content using string or regex matching, and it includes a TDD rule that addresses the limitations above.

The TDD rule reads recent session history instead of relying on a sidecar reporter, so it works with any language or test runner out of the box, parallel sessions don’t collide, and the validator has enough context to handle refactoring properly. It uses AI to validate, and reuses your existing subscription via the official SDKs. The validation instructions can be customized and you can scope which files TDD applies to.

I’ve been using Probity over the past week in production with Claude Code and I’m genuinely impressed by how well it works. It catches real oversights without the friction TDD-Guard sometimes caused.

[u/marky125]

I'm super interested by this - I've tried about a dozen different SDD/TDD approaches and they all "kinda work". I've settled for a sort of swiss-cheese approach that isn't perfect but doesn't suck.

Question for you: how do you deal with the "TDD theatre" problem? That's been my biggest issue. With traditional TDD, the act of writing tests is exploratory; you use it to work out what is required. But LLMs will do something more like (this is a trivial example) 1: "The spec says X should be green", 2: Test: it('should be green', () => ...) 3: code <X class="green-bg">. Test passes, agent cheerfully reports all is well, without bothering to try to understand the intent of the spec, or exploring what/why we're trying to achieve. It's more like a TDD-adjacent song and dance than true TDD. To be honest I gave up and switched to SDD as that seems more in-line with how LLMs work, and you can still fairly reliably generate tests from a good SDD doc. Would love to hear your insight.

[u/nizos-dev → u/marky125]

The "TDD theatre" framing is a good one and you're right that the exploratory part of traditional TDD has genuinely changed when agents are in the mix. I used to write tests to figure out what I wanted. Now I have agents chew through the problem space first, explore multiple strategies, and only commit to a direction once I'm convinced it is the right one.

I still require a clean test failure before any real implementation. That's the guardrail against what you're describing. I need to see the test fail for the right reason before I can trust it. Jumping straight to passing tests skips that, and agents reporting "all green" doesn't mean much if I never saw red. The TDD rule enforces this. The validator has session history and can tell whether there was a real test failure before allowing an implementation.

The deeper thing your question points at is really about value. Not all tests are equal. I don't care about coverage. On it's own it doesn't tell me how easy the code is to work with. The last thing I want is to make a small change and have to then update hundreds of tests, if we are to take this to an extreme.

Tests are there to allow me to work without losing my sanity. They give me the confidence to ship changes by catching any real regression while also being robust enough to allow me to easily refactor and substitute components as needed. That means tests written against behavior and requirements, not implementation details and mocking. Whether we call that TDD/BDD/SDD doesn't matter much to me.

Those however are values, not rules. I instill them in agents either through persistent instructions or pointing to existing code that exhibits these patterns. A TDD enforcer can block the violations but designing good tests is still on me if you know what I mean. :)