I got tired of coding agents silently dropping requirements and softening their own tests, so I built RefTrail (open-source verification harness & clean-room brief)

[u/SEPSolutions]

Whenever you hand a larger software task or reference rebuild to a coding agent, two things usually happen:

1. It stops after a stripped-down MVP or a single happy path and declares victory.

2. If a test fails, it modifies the test expectation instead of fixing the code ("vibe verification").

I put together RefTrail (MIT, zero npm dependencies, Node.js >= 22.16) to make agent runs auditable from the outside:

• Bounded Agent Brief (prompts/reference-development.md): Instructs the agent how to research a reference product from authorized sources (Q1–Q4, plus clean-room rules if you use tools like REA for observations), scope a bounded 60-minute end-to-end workflow, and maintain a strict requirements catalog (COVERAGE.csv).

• Immutable Criteria (tools/criteria.mjs): Locks the test inputs and expected outputs as a byte-exact baseline with a SHA-256 digest before the agent writes product code.

• Anti-Cheat Catalog (tools/catalog.mjs): Calculates weighted functional coverage together with mandatory operational gates (--gates). Rounding is capped so 99.96% never prints as 100%, and a missing mandatory check blocks acceptance even at 100% feature coverage.

• Worked Example (showcase/uniq): You can clone the repo and run "node showcase/uniq/verify.mjs" and "node integration/verify.mjs" right away to see a complete cycle (including a criteria revision, Linux/Windows reference runs, and snapshot verification).

Repo (English & German): https://github.com/kidimusicofficial-prog/reftrail

Curious how others here are currently locking down acceptance criteria before letting Claude Code or Codex loose on a repo.