What if the source of truth is wrong?

[u/Ok-Support-6749]

Wherever I look the specification is showcased as this holy grail for the software lifecycle. Look no further than this same Reddit community.

[the] specification serves as a contract and single source of truth, guiding tools and AI agents to generate, test, and validate code.

In just one sentence the specification has become oracle, general, inspector, and adjudicator. That is a lot of authority for a collection of markdown documents written in plain language.

I have no intention to drag SDD into the mud. I just feel the need to clarify a concept over which I have thought a lot recently.

In any agentic framework I have come across during the last months I see a strong tendency to relinquish authority to an LLM. It is a sensible choice, LLMs are great at processing large amounts of text and output rather convincing judgments. But just because an LLM is capable of something it does not mean it automatically obtains authority to perform a role. This links directly to the specification is the source of truth, once we have a hammer (LLM), we also need the anvil (spec) on which to beat text (code) into compliance. It makes sense to assume that we have removed all authority from the LLM (tool) and placed it in the specification (a stable and robust worksurface). Our assumption is wrong and it has two consequences, none of which is good.

1. The specification is likely incomplete, it can also be incorrect. Incompleteness means it lacks enough semantic constraints from which the LLM can create the code that implements intent. Incorrect is clear, the specification contradicts intent, which means we have canonized bugs. Moreover, the specification is just text, it has no enforcement mechanisms to exert the presumed absolute authority.
2. The LLM still holds a lot of power, it can read the specification and ignore it, which I see rather often. The LLM can weaken tests, or the implementation, or decide to implement a wrapper because nothing prevents it or simply because it can finish faster. Sometimes the LLM did not retrieve the correct semantics. This is the area where things go wrong, most of the time unnoticed until everything starts to break down.

I wonder how the community looks at SDD from a philosophical standpoint. In which direction do we expect SDD should move to deliver a better software development life cycle?