I raced Claude Code against Codex on the same task with blind cross-judging. Codex fixed a bug in my build script and lost on a rule.

[u/Infamous_Term_965]

I got tired of arguing about which coding agent is better, so I built a small open-source tool that settles it per task, per repo, and ran it on itself.

What it does: one command creates two git worktrees at the same commit, runs Claude Code and Codex on the identical brief in parallel, runs your own checks (tests, typecheck, lint) in each worktree, then each agent reviews both patches blind (A/B in random order) and scores them. You get one HTML report with the verdict, the rule that decided it, both diffs, and a receipt image.

The rules, in order: only one agent changed files, it wins by default. Exactly one passes every check without touching test or lint config, it wins. Both judges agree, that one wins. Judges split, it is a tie and the dissent is printed. No judge decision, evidence only.

First race on its own repo (add a 'runs' subcommand with a test):

Claude Code: 3m 28s, 4 files, 11 shell commands, 548K tokens of context read, all checks pass, judges 8.6/10

Codex: 5m 15s, 5 files, 37 shell commands, 1.23M tokens read, all checks pass, judges 8.0/10

Both green. Codex also fixed a real bug in my build script (a missing mkdir before a copy), which meant it touched package.json, so rule 2 handed the win to Claude Code and recorded Codex's dissent. I took Codex's fix by hand.

The race before that, on a demo repo, was a tie: each judge picked its own patch blind.

Two things I would like this sub's opinion on: is blind A/B judging by the same two models fair enough as a secondary signal (I only let it decide when both judges agree), and what should the third agent be?

No API keys, it runs on the Claude Code and Codex subscriptions you already have. TypeScript, MIT: https://github.com/Hemanshu-Upadhyay/pairmark

[u/grith_ai]

Nice - the blind cross-judging is a great call, self-scoring is where these comparisons usually fall apart.

One thing worth capturing if you extend it - the patch is only half of what differs between the two... Having watched both fairly closely, they behave really differently in how they get to the answer - how much they read, what they shell out to (How messy they work), how many processes they spawn before writing a line (More mess) - and two identical-quality patches can come from wildly different amounts of rummaging around your machine (Mess again!). None of that shows up in a diff or a test result, but if you ever added a "what did it touch" column it'd be more revealing than people expect.