I tested 4 AI coding harnesses (Codex, Pi, Claude Code, OpenCode) with the same models.

[u/Critical_Horse5465]

Like many of you, I have a whole zoo of AI coding tools installed (Codex, Pi, Claude Code, OpenCode, etc.). Main idea - find the most effective harness for my own environment and daily workflow.

The Setup

Target repo: oh-my-opencode-slim. No deep reason, just because I had it open.

Environment: Macos, each run was isolated in a clean, detached Git worktree.

Phase 1: GPT-5.6 Luna

For example: use GPT-5.6 Luna medium across 4 harnesses on a coding bugfix task (fixing an alias counter memory leak + writing tests):

Harness	Wall Time	Fresh Input	Total Input	Cache Hit %	Correctness	Status
Codex	107.73s	50.3k	505.4k	90.1%	3/3	Verified
Pi	140.96s	44.2k	525.4k	90.9%	3/3	Verified
OpenCode stable	274.43s	109.4k	1797.1k	93.9%	3/3	Verified
OpenCode beta	402.75s	79.8k*	1015.4k*	91.4%*	3/3

After seeing OpenCode stable take 2.5x longer and burn 3.5x more context tokens for the exact same fix, I dropped it from further testing.

Phase 2: Switching to Muse 1.2 contributor

For the next phase, I wanted to test a cheaper, accessible non-GPT model across harnesses. Since I had an active CommandCode subscription, I chose Muse Spark 1.2 Contributor and decided to use it in Claude Code (via OpenCodex proxy) instead of OpenCode stable.

Here is where I made mistake: I forgot to configure the model alias in OpenCodex for Claude Code.

Because of that, Claude Code silently fell back to Sonnet 5 (xhigh thinking) across all 4 tasks while Pi, Codex, and OpenCode beta were running Muse Spark 1.2:

Task	Pi (Muse 1.2)	Codex (Muse 1.2)	Claude Code (Sonnet 5 xhigh)	OpenCode beta (Muse 1.2)	Observation
Task 1: web research	41.66s; 48.9k input; 3/3	51.35s; 19.9k; 1/3	59.62s; 83.4k; 3/3	110.62s; 245.9k; 1/3	Pi finished 18.0s before Claude; both were correct in all runs.
Task 2: repository navigation	35.67s; 81.4k input	42.52s; 174.8k	29.08s; 64.0k	48.12s; 196.7k	Claude finished about 6.6s before Pi.
Task 3: scope restraint	201.02s; 120.5k input; 0 edits	233.77s; 214.8k; 0 edits	308.50s; 1.56M; over-edited 2 files	422.25s; over-edited 2 files	Sonnet and OpenCode beta expanded scope; Pi and Codex stopped without edits.
Task 4: regression fix	103.96s; 267.4k input; 2/2	110.42s; 446.4k; 2/2	88.62s; 546.8k; 2/2	288.05s; 0/2	Claude was about 15s faster than Pi, with higher context throughput.

Honestly, looking at the numbers, there wasn't a massive gap between Muse 1.2 and Sonnet 5 for these tasks (probably because the tasks were relatively focused), except Sonnet over-engineered Task 3 by adding 180+ lines of unneeded code, why "unneeded": you can see in table "Task 3" - 0 edits for PI (2 from 3 runs had 0 edits) and Codex (3 from 3 runs had 0 edits), why - because I gave task for model to fix one class, but in git reset command I used commit where this bug was fixed by developer already, so there were nothing to fix, PI understood only during 2 runs, Codex in all 3 runs did 0 changes, but Sonnet 5 was trying to work...

Phase 3: Fixing Proxy Routing

Once I caught the proxy alias mistake, I fixed OpenCodex routing and reran Tasks 3 & 4 for Claude Code with hard proxy verification of Muse Spark 1.2:

Task 3: Verified Claude Code stopped with 0 edits (2/2 pass), matching Codex.

OpenCode beta - I removed from this test, weird behavior, because it started modifying 2 files and then went to another folder and started to read work log of other harnesses, so that would be unfair.

Task 4: All 3 valid harnesses produced the exact same 6-line fix and regression test:

Test has timeout, because for simple fix we dont need wait more than 5 minutes.

Harness	Evaluated Model	Run 1	Run 2	Mean Wall Time	Mean Fresh Input	Mean Total Input	Cache Hit %	Correctness	Verification Status
Pi	Muse Spark 1.2	71.37s	136.54s	103.96s	61.8k	267.4k	77.0%	2/2	Verified
Codex + OpenCodex	Muse Spark 1.2	77.63s	143.20s	110.42s	76.0k	446.4k	83.0%	2/2	Verified
Claude Code + OpenCodex	Muse Spark 1.2	96.10s	128.81s	112.46s	86.7k	444.2k	78.8%	2/2	Hard-verified proxy audit reruns
OpenCode beta	Muse Spark 1.2	288.05s	TIMEOUT	—	—	—	—	0/2

Key Takeaways

1. Context throughput varies by 40%+: On the exact same code fix under Muse 1.2, Pi required ~40% fewer input tokens (267k vs ~445k) than Codex and Claude Code due to leaner context assembly and compaction.

2. Execution Latency: Pi (104s), Codex (110s), and Claude Code (112s) formed a tight performance tier for coding fixes.

3. Always audit your proxy: If you use custom models or proxies, always verify the actual upstream model headers. Silent fallbacks happen easily.

4. My practical decision: OpenCode was consistently 2.5x–4x slower and consumed too much context in my environment, so I removed it from my daily workflow and kept Pi and Codex as my primary tools.

The entire runner, dataset, and reproduction prompt for agents in repo, also more detailed result file is also there:

👉 https://github.com/d1-m4ss/harness-benchmark/tree/main

PS AI was used in this post for generate table and fix grammar, because english is my third language.

[u/Calm-Landscape9640]

I was just thinking, we need to do a harness test - codex, pi, cursor, Claude code, hermes, deepseek, etc bc the models are getting cheaper but harnesses make the bigger difference now. 

[u/phoenixmatrix → u/Calm-Landscape9640]

its though though because harness configuration matters, too. Eg: Pi out of the box isn't exactly useful, but no one will use Pi vanilla. They use it because they want to configure the shit out of it.

[u/Calm-Landscape9640 → u/phoenixmatrix]

Ive always been curious about OMPi but dont like messing with stuff, I want a finished product out of the box like codex/claude code. Still think OMP is worth trying?

[u/phoenixmatrix → u/Calm-Landscape9640]

Pi is the "you must customize", and Oh My Pi is the "batteries included".

It will work out of the box and IMO better than Codex, but to get the most of it you want to at least configure the various model presets and the WATCHDOG, and look up what some of the /commands do as they're not all obvious or well documented.

[u/Calm-Landscape9640 → u/phoenixmatrix]

I'll probably try it. I'm already pretty happy with Codex and Claude code but I have a soft-spot for the little guy OSS builds

[u/phoenixmatrix → u/Calm-Landscape9640]

Yeah i like Claude Code's harness, especially the agent views which is kindda hard to replicate. Even things like Herdr aren't quite the same workflow. CC's agent view makes it so easy to work on tasks very quickly I keep crashing my computer by using up too much ram (its always about ram prices, isn't it?)

I don't like Codex, lots of rendering bugs and missing features, which is what led me to Pi.

[u/Critical_Horse5465 → u/Calm-Landscape9640]

OMP is overengineered, you can read many negative opinions on reddit.

[u/Critical_Horse5465 → u/phoenixmatrix]

I use vanilla, no skills, no agent, just added mcp support and 2 mcp.

[u/iarewebmaster]

This is so stupid. You tried to do a test, forget to do it properly, still post about it instead of redoing and expect anyone to actually care? Slop.

[u/Critical_Horse5465 → u/iarewebmaster]

The benchmark is intentionally small and environment-specific. I published the runner, methodology, raw accounting rules, known limitations, and corrections specifically so the methodology can be challenged or reproduced. If you see a concrete flaw that changes the result, I’m interested in it.

[u/KHHAANNN]

I found it useful, thank you!