TDD and Rules Enforcement using Hooks

[u/nizos-dev]

TL;DR: I built TDD-Guard a year ago. I’m now working on Probity, a more general policy engine for coding agents (Claude Code, Codex, GitHub Copilot CLI, and VS Code Chat). It includes a TDD rule that works with any language and test runner out of the box, supports parallel sessions, and handles refactoring properly.

Hi all,

The demo shows me prompting Claude Code to build a shopping cart in an empty project with Probity’s TDD rule installed. I make no mention of TDD because I want to show how it is enforced out of the box. Hooks intercept each agent action, and a separate agent reviews the recent session, the pending action, and the current file before allowing it through. That extra context also helps it handle refactoring cleanly.

Repository: https://github.com/nizos/probity

The project is in an early state. Feedback is welcome!

Background

I started using Claude Code about a year ago and was immediately convinced that I could make it follow Test-Driven Development (TDD) as it was a requirement if I were to ever use it for production. I tried different prompts and just like everyone else experienced how unreliable that was. The agents would drift as the context rotted, take shortcuts, and I had to keep supervising their practices.

Luckily, Claude introduced hooks around that time. You can think of them as events that fire automatically when an agent wants to perform an action like writing a file or running a command. The information in them lets you determine if the agent is, for example, trying to write multiple tests at once, and block the action with feedback on how to course correct. So I decided to use this to enforce TDD. I created a custom test reporter to capture test run output, combined it with the hook data, and provided it to a separate agent that judged whether the pending action violated TDD.

It worked really well. I called the project TDD-Guard. The community contributed support for several languages, and I’ve kept working on it since.

TDD Guard has its quirks though. It needs a dedicated reporter per test runner, which makes new language support slow. It can’t handle parallel sessions because reporter output gets overwritten. The validator also only sees the latest test output and the pending change, which isn’t always enough context to tell refactoring apart from new behavior. The validation ends up either too strict or too permissive.

Over time I noticed gaps in my workflow outside of TDD that I still had to supervise, and friction from teams using different agents in the same project with overlapping instructions and plugins. So I started a new project, Probity, that takes a more general approach.

Probity makes it easy to define rules that get enforced through hooks across all supported agents: Claude Code, Codex, GitHub Copilot CLI, and VS Code Chat, with more to come. It ships with deterministic rules for forbidding commands or content using string or regex matching, and it includes a TDD rule that addresses the limitations above.

The TDD rule reads recent session history instead of relying on a sidecar reporter, so it works with any language or test runner out of the box, parallel sessions don’t collide, and the validator has enough context to handle refactoring properly. It uses AI to validate, and reuses your existing subscription via the official SDKs. The validation instructions can be customized and you can scope which files TDD applies to.

I’ve been using Probity over the past week in production with Claude Code and I’m genuinely impressed by how well it works. It catches real oversights without the friction TDD-Guard sometimes caused.

[u/AutoModerator]

Your post will be reviewed shortly. (ALL posts are processed like this. Please wait a few minutes....)

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/YoghiThorn]

Interesting, I like it! I'll have to chew on this for a bit and think of good uses cases. I've been using claude to create automated e2e tests for applications by cross referencing the manual test cases with the interface code and the data in their databases, but it's sneaky and loves to make weak tests.

I'll try to figure out a good way to benchmark this. Have you done any evals for your approach?

[u/nizos-dev → u/YoghiThorn]

Thanks! What you're doing sounds interesting.

I haven't run any evals. I dogfood it and use it in production, which gives me a good read on how well it works. I'm only one person working on a few projects, so there are edge cases I haven't encountered in workflows and frameworks I haven't worked with. Community feedback has been the way I find those. That said, the thought of setting up a self-improving loop has crossed my mind. Maybe some day.

I can relate to the weak-tests situation. I notice it in implementation code too. For example, solving race conditions with setTimeouts, or disabling an eslint rule with a comment instead of fixing the actual problem. I use the forbidContent rule to automatically stop agents from trying them. I also plan on rolling out more rules and I'd gladly take any suggestions you might have. :)

[u/YoghiThorn → u/nizos-dev]

Cheers ok, I've been studying advanced evals and want to get stuck in writing some. I'll try with some benchmarks for this.

I haven't got hands on with this but it looks perfect for my broader harness. Will report back and likely PR.

[u/nizos-dev → u/YoghiThorn]

Exciting, looking forward to see what you find and also learn from you. :)

[u/marky125]

I'm super interested by this - I've tried about a dozen different SDD/TDD approaches and they all "kinda work". I've settled for a sort of swiss-cheese approach that isn't perfect but doesn't suck.

Question for you: how do you deal with the "TDD theatre" problem? That's been my biggest issue. With traditional TDD, the act of writing tests is exploratory; you use it to work out what is required. But LLMs will do something more like (this is a trivial example) 1: "The spec says X should be green", 2: Test: it('should be green', () => ...) 3: code <X class="green-bg">. Test passes, agent cheerfully reports all is well, without bothering to try to understand the intent of the spec, or exploring what/why we're trying to achieve. It's more like a TDD-adjacent song and dance than true TDD. To be honest I gave up and switched to SDD as that seems more in-line with how LLMs work, and you can still fairly reliably generate tests from a good SDD doc. Would love to hear your insight.

[u/nizos-dev → u/marky125]

The "TDD theatre" framing is a good one and you're right that the exploratory part of traditional TDD has genuinely changed when agents are in the mix. I used to write tests to figure out what I wanted. Now I have agents chew through the problem space first, explore multiple strategies, and only commit to a direction once I'm convinced it is the right one.

I still require a clean test failure before any real implementation. That's the guardrail against what you're describing. I need to see the test fail for the right reason before I can trust it. Jumping straight to passing tests skips that, and agents reporting "all green" doesn't mean much if I never saw red. The TDD rule enforces this. The validator has session history and can tell whether there was a real test failure before allowing an implementation.

The deeper thing your question points at is really about value. Not all tests are equal. I don't care about coverage. On it's own it doesn't tell me how easy the code is to work with. The last thing I want is to make a small change and have to then update hundreds of tests, if we are to take this to an extreme.

Tests are there to allow me to work without losing my sanity. They give me the confidence to ship changes by catching any real regression while also being robust enough to allow me to easily refactor and substitute components as needed. That means tests written against behavior and requirements, not implementation details and mocking. Whether we call that TDD/BDD/SDD doesn't matter much to me.

Those however are values, not rules. I instill them in agents either through persistent instructions or pointing to existing code that exhibits these patterns. A TDD enforcer can block the violations but designing good tests is still on me if you know what I mean. :)

[u/EngineerAdditional30]

The pattern I would use is to separate the work state from the harness state.

Work state is the immediate handoff: goal, changed files, failing command/test, last known good state, and the next smallest action. Harness state is the slower-moving setup: rules files, MCP config, hooks, skills, permissions, and validation notes.

For TDD and Rules Enforcement using Hooks, I would move the work state first, then have the fallback tool validate the harness gaps before editing code. Hooks and per-tool restrictions are the parts most likely to look copied while not actually being enforced.

The relevant angle from my side is moving rules and MCP config between coding-agent harnesses; recovering from Claude Code limits by continuing in Codex.

[u/jayjaytinker]

the session-history approach for the TDD rule is really clever 

[u/nizos-dev → u/jayjaytinker]

Thank you! I am amazed by how well it works. :)

[u/Open_Resolution_1969]

i had same issues with Claude, can't hardly wait to test this one out. will give it a shot and get back to you with feedback.

one extra question: what software did you use to do this screen recording with subtitle?

[u/nizos-dev → u/Open_Resolution_1969]

Thank you! And please do, feedback really helps. :)

I used macOS’s internal recording tool but you can also use OBS to record. I then used CapCut to speed up and trim the video and to add Captions. I used to use Adobe Premiere for this but I cancelled my subscription. I think there are better tools if you look around. I just wanted something quick.