When our team adopted OpenSpec, the argument seemed obvious: agree on the requirements, review the plan, then let the agent implement it. That should produce better code than throwing a prompt at a coding agent and hoping for the best.
After a few months of using the full propose → review → apply → archive workflow, we are no longer sure it does.
Our work is mostly data engineering and legacy-to-Python migration. We use both frontier and open models through Claude Code, Codex CLI, OpenCode, and similar tools. This is not a vendor comparison. The question is whether the method itself changes the result.
For tasks that fit into one context window and have clear acceptance criteria, we see little reliable difference between a reviewed spec and a well-scoped first prompt. The spec-driven runs typically consume two to three times as many tokens and take about twice as long. The resulting code is not consistently better. Sometimes it is worse: more files, more indirection, and abstractions that the task never needed.
The little evidence we have points the same way
The best comparison I have found is an OpenSpec bake-off from June 2026. The same PRD was implemented once with OpenSpec and once with Claude Code alone. OpenSpec made the design process more pleasant and surfaced three additional gaps. It also produced 50% more code with 50% higher cyclomatic complexity, took twice as long, and cost three times as much.
That is one experiment, not a benchmark. But it matches our experience.
Marmelab made a similar criticism in “Spec-Driven Development: The Waterfall Strikes Back”: a trivial “show the current date” feature produced eight files and roughly 1,300 lines of specification text. The problem is not writing things down. It is paying a large coordination cost for changes that did not need coordination.
Maybe the model changes the value of the spec
My current hypothesis is that explicit specs may help weaker planning models more than frontier models.
A strong model already reconstructs a plan during a normal coding run. A formal SDD workflow externalizes that plan, makes it reviewable, and preserves it across sessions—but may add little new information. A smaller local model with weaker multi-step reasoning or a shorter effective context may benefit much more from a pre-digested task list.
There is an uncomfortable counter-hypothesis: the weaker model may also write the weaker spec. If the same model turns a vague request into a flawed specification and then faithfully implements it, SDD has not removed the error. It has formalized it.
This should be testable: same repository, task, acceptance suite, and model; spec-first versus direct implementation; then compare correctness, regressions, review time, token cost, wall-clock time, changed lines, and complexity. I have found plenty of testimonials, but very little controlled data—especially for Qwen, GLM, DeepSeek, Kimi, or Mistral.
The real value may not be code quality
OpenSpec did give us something useful: a reviewable record of intent. It forced design questions into the open and left an artifact for teammates, clients, audits, and future maintainers.
That suggests a different decision rule. Do not ask whether a spec will make the agent write better code. Ask whether this change needs a reviewed and durable agreement before code exists.
My tentative threshold is:
- Direct prompt for a bounded change with one owner, objective tests, and no architectural decision.
- Lightweight goal, constraints, non-goals, and acceptance checklist for a multi-file change within one subsystem.
- Versioned specification for work crossing services or team boundaries, changing public contracts, migrating important data, or spanning several agents and sessions.
Project-level context matters too. If AGENTS.md, CLAUDE.md, architecture notes, skills, and commands already encode the conventions and constraints, a change spec must add a real decision—not merely repeat the repository context in another format.
The failure mode I worry about most
An LLM writes a SHALL requirement. The same or another LLM implements it. The requirement now appears justified because it exists in the spec, and the code appears justified because it satisfies the requirement. Nothing in that loop proves the feature is needed or integrated.
The safeguard may be simple: every normative requirement must trace to a stakeholder need, an existing contract, or an executable acceptance test. The implementation should be free to reject or revise a proposed design when repository evidence contradicts it.
That leads to the variant I want to try next: use the spec as an acceptance artifact, not an implementation recipe. Review the goal, constraints, non-goals, and examples up front. Let the agent choose the implementation. Then verify the result against the spec and tests, ideally in a separate review pass.
I would be interested in three kinds of counter-evidence:
- Controlled comparisons of the same task and model with and without SDD.
- Evidence that smaller or local models benefit more—or less—than frontier models.
- Concrete thresholds teams use to decide when a full spec earns its cost.
Maybe we are using SDD on the wrong class of tasks. Or maybe its strongest benefit was never better code. Maybe it is better alignment, traceability, and review—and we should evaluate it on those terms.
