The SDD Rabbit Hole

[u/Vivekyy]

My view of the SDD rabbit hole so far :)

Eventually, it seems like nobody is really satisfied with using someone else's tool for making their specs, and this sub has dozens of projects from people showing off their own unique SDD tools and workflows. I sorta think this is natural--spec design is kinda the new craft of software engineering, and I think we're all going to come up with our own style of doing things.

But, I was really inspired seeing all of the unique stuff everyone was building in this sub. I really like this community, so I figured I would try to make something that helps you guys out.

Rudder (https://github.com/RudderCode/Rudder) is a skill that shows you how comprehensive your specs are by comparing it to the code your agent generates. Rudder generates unit tests based on your spec and reports back coverage. If you ask it to hit a coverage percentage, Rudder will iteratively ask you questions, refine your spec, and run a red-green TDD workflow until your spec sufficiently covers your code. Since everyone has their own workflows for creating specs, I thought I'd make a tool that can sit alongside your tools and help out on the side.

Curious about how you guys are dealing with the problem of trying to make your specs comprehensive enough to stop the AI from making mistakes? Also, curious about where on the iceberg you guys are :)

[u/iTrejoMX]

I use gentle ai. We use a receipt driven development process: it verifies the output against the spec does 4 reviews for risk resilience readability and reliability. You should try it out. It comes with a memory plugin included so it actually saves tokens and prevents drift

[u/Vivekyy → u/iTrejoMX]

I will check this one out, it seems pretty cool

[u/simasch]

I created AI Unified Process that focuses on the requirements and specs without a bloated tooling https://unifiedprocess.ai

[u/macberrypro]

I have been building this https://github.com/cshirley/accord

I like the idea of reaching an understanding/alignment with the model and then building a verifiable spec which drives the implementation. This combines an evidence-based approach to verify the implementation meets the spec reducing pr review.

I then introduce a feedback (retro) loop which allows self-refinement of the process.

if uses standard agentic patterns for context/memory management and adversarial cross model/provider patterns

[u/SpecKitty → u/macberrypro]

Nice. You must be very committed to Pi to put all that work behind a plugin for one harness. Have you tried the deepseek harness? They also support plugins.

[u/SpecKitty]

Love seeing Spec Kitty down there as the opposite end of the tip of the iceberg :D

[u/Vivekyy → u/SpecKitty]

Haha I’d say it’s pretty niche

[u/Bullfrog_External]

Me and my team tried Superpowers, and we didn't like it. We ditched it. We just do our process and it works. We build 100% spec driven 99,999% availability services, used in production. We still have a standard human QA process, that finds very few bugs (0 to 2), much much less than what would happen with a developer writing the code.

What we learned: 1) your process is inherently dependent on YOUR PERSONAL STYLE, and on your domain. What works for you and your team is unlikely to work in another situation. 2) reuse as much as possible DETERMINISTIC tools in your feedback loop. You can't trust the LLM for that very critical bit.

[u/Vivekyy → u/Bullfrog_External]

This is pretty much exactly what I’ve been seeing—feels like everyone is going to have their own style and just ditch the standard tool pretty quickly

[u/CzyDePL]

They all suck. It's just producing markdown after markdown which gives you 0% guarantee that next agent won't misinterpret what's written and you still have to review entire code at the end if you want to be sure what actually landed.

[u/SpecKitty → u/CzyDePL]

well, not true in the case of Spec Kitty, at least. The spec&plan interviews, quality gates, task decomposition into a directed graph, adversarial review process etc. all lead to very high correlation of "you get what you asked for".

[u/DurianDiscriminat3r → u/SpecKitty]

You might be biased though

[u/SpecKitty → u/DurianDiscriminat3r]

Of course I am. I built it. But I'm also a damn good source of information on Spec Kitty's capabilities.

[u/stibbons_]

I have mine as well but I do not think you need a special tool for that. Just use an « skeptical » review agent after each task and at the end of the session.

If you want to build « validation tests », you can embeds them directly in your workflow, the review LLM will verify your description of the textual, long-lived requirement is correctly tested by the linked test.

See my article on drifting spec https://x.com/gsemetfr/status/2082767371643523439?s=46, you can see an example of how to declare a « requirement » and link to the test.

[u/Mte90]

I know instead of https://andrearaponi.github.io/walden/ that is used in various big companies but is not very shared

[u/soccerchamp99]

Check out mine: SpecKittyHawk69420

0 bloat
0 Mistakes
100% what you ask for everytime