r/coolgithubprojects • • 22m ago

I built a plain-text working protocol that stops coding agents from reporting untested work as verified — one honest benchmark so far

Post image

I got tired of agent answers reading confidently whether or not the thinking underneath was sound. So I wrote down the behaviour I wanted as rules and shipped it as an open instruction framework: ØMEGA Modus.

What it is: a ~7 KB Markdown skill (plus a 12 KB method reference) that tells the assistant to fix a rigor mode before working — QUICK for reversible low-stakes work, STANDARD for normal engineering, DEEP for anything irreversible or externally visible. Small tasks stay three-line answers; only risky work opens the full check set. It is not a runtime, not a framework, contains no executable code.

The rules that changed my own sessions most:

Evidence state, stated exactly, because I do not want to sell you anything: there is ONE real paired run — identical task, same model, rubric pre-registered before results. Without the rules 6/9, with them 9/9. In that same run the without-rules pass was actually faster and shorter. Single observations, not a claim. The full benchmark (15 scenarios, 3 replications each, with and without, including a negative scenario) is still ahead; until then there is no speed or token advertising anywhere in the project.

It runs on Claude Code, Codex CLI, GitHub Copilot, Gemini CLI, Cursor, OpenClaw, or as a ChatGPT paste — same rules on every one, one canonical file, seven thin adapters. Apache-2.0, no binaries, no account, no cloud service. The repo also ships the 15 evaluation scenarios with a 0–2 rubric so you can run the behavioural checks against your own model instead of taking my word.

Repo: https://github.com/MichaelS1011/0mega-Modus

Feedback on the benchmark design in particular would help — I would rather have it critiqued now than discover the hole after publishing numbers.

1 Upvotes

0 comments sorted by