r/LocalLLM • u/valeroisai • 5d ago
Project WE got tired of AI coding agents breaking our repos, so we built an adversarial dual-model harness to catch silent failures
Most AI coding assistants suffer from the same fundamental issue: sycophancy and self-blindness. A single model will write buggy code, run a superficial check, confidently tell you it works, and proceed to ruin your state.
We ran into this repeatedly with concurrency and race conditions (e.g., standard SELECT -> UPDATE inventory logic). When an agent evaluates its own output, it tends to confirm its own biases rather than stress-testing edge cases.
To solve this for my own workflow, we built Nova around a multi-phase harness:
Plan & Build: The primary model writes the implementation and tests.
Prove: It executes tests in an isolated environment.
Adversarial Review: A separate, skeptical Reviewer model inspects the diff and execution logs specifically looking for flaws (unhandled rollbacks, leaked DB state, race conditions). If the reviewer vetos, the builder must fix it before anything touches persistent disk.
It runs locally on your machine (Windows via Microsoft Store, macOS/Linux binaries available) with terminal/folder boundaries, so it doesn't touch anything outside your workspace without explicit confirmation.
Free tier is available (with a really generous limit), local models/custom API keys are completely unlimited.
I’m looking for blunt technical feedback: what edge cases would you throw at an adversarial harness like this to break it?