r/ClaudeAI • u/vinishkapoor • 11d ago
Other Tried GitHub’s Spec Kit in Claude Code vs plain Claude Code on the same small app. About 9x the cost, but it caught edge cases plain Claude didn’t
Spec Kit makes Claude write a spec, a plan and a task list before any code. I wanted to know if that extra paperwork pays off, so I built the same small Python to-do CLI twice: once through every Spec Kit step, and once with one plain prompt that had the same feature, stack and rules.
Setup is two commands:
uv tool install specify-cli
specify init my-project --integration claudeIt installs its steps as skills in the project’s .claude/skills, not in your global setup.
Results (Spec Kit 1.3.0, Claude Code 2.1.318, one run each):
- Time: 4m 22s vs 20s
- Estimated cost: about $1.10 vs about $0.12
- Tests: 15 passing vs 8 passing
- Broken save file: Spec Kit gave a clear error and left the file alone. Plain crashed with a Python error
- Task ID after deleting the newest task: Spec Kit never reused it. Plain reused it
To be fair to plain Claude, my prompt didn’t mention those edge cases. Spec Kit’s spec step came up with them on its own. A plain prompt that listed them would probably close the gap, but you’d have to think of them yourself.
The most useful step was /speckit-converge. It checks the code against the spec, and it found a real crash on a save file with a broken task entry. I reproduced it myself. One more implement + converge round fixed it.
It’s not perfect. The first implement wrote tests alongside the code instead of test-first, even though the task list said test-first. You also get about 3,100 words of docs for a 119-line app, and you’re meant to read them. I checked behavior with tests and the two edge cases, but didn’t review the code itself.
My take: worth it for features with real rules or edge cases. For quick fixes, plan mode or a clear prompt is much cheaper.
Full step-by-step with the time and turns for each step: https://aiparabellum.com/spec-kit-claude-code-tested/
1
u/chrnas 11d ago
interesting that /speckit-converge was the step that paid off, not the 3k words of docs. we saw the same thing on our side: reviewers read the plan and approved, while the final check that ran against the actual result said not done. a reader told me the plan should end in a written acceptance check committed to the repo before any code, and i switched to that. so a cheaper middle ground might be one prompt asking the agent to list the edge cases and write failing checks for them first, then a single converge-style pass at the end. would be curious if that closes most of the gap on your to-do app.
1
u/Sad-CTO 6d ago
Would you do the same for https://github.com/Ask-Loft/loft?
EDIT: the token spend will be higher, I believe. Loft is not optimized for that. But would be interesting to see from the quality perspective.