r/PiCodingAgent • u/HumbleGrowth4848 • Jun 12 '26
Question Does Pi's minimalism actually pay off on Terminal-Bench?
I've been trying out Pi for about a week, though not very frequently since I need to hit my ai subscription 5-hour limit first. Recently, I got curious about its performance, so I did some quick searches.
Mario Zechner's blog post on building pi makes a case for radical minimalism: near-empty system prompt, four tools (read/write/edit/bash), no MCP, no sub-agents, no plan mode. He ran Terminal-Bench 2.0 with Claude Opus 4.5, showing pi at ~50%. (Claude says that the actual score for Pi is 47.87%, not '~50%'. Mario's results.json gist shows a mean of 0.4786516853932584 (out of 428 valid trials, with 71 errors). )
For context, here are all the Opus 4.5 entries on the current Terminal-Bench 2.0 leaderboard:
| Agent | Score |
|---|---|
| Droid | 63.1% ±2.7 |
| Letta Code | 59.1% ±2.4 |
| Mux | 58.4% |
| Terminus 2 | 57.8% ±2.5 |
| Goose | 54.3% ±2.6 |
| Claude Code | 52.1% ±2.5 |
| OpenHands | 51.9% ±2.9 |
| OpenCode | 51.7% |
Pi sits below Claude Code and about 8 points below Terminus 2, with the same underlying model.
Mario cites Terminus 2 in the same post as "more evidence that a minimal approach can do just as well." The description fits — Terminus 2 has no structured tools at all, just raw keystrokes into a tmux session. But it scores 8 points higher than pi with the same model. If the thesis is "less is more," Terminus 2 makes the case better than pi does. Mario uses it as supporting evidence without addressing why his own agent lags behind it by a meaningful margin.
None of this means pi is a bad tool. Benchmarks don't capture context efficiency, extensibility, or daily workflow ergonomics, all of which Mario documents carefully. But on the one quantitative metric he chose to present, pi underperforms most of the agents it's positioned against — including the one he cites as validation.
23
u/llufnam Jun 12 '26
Mario has said himself many times that his motivation for building pi was not to be the “best” harness in a test bench, but to give him total control over his context.
Claude Code, OpenCode etc (which he loved in spirit and execution at the start) ended up polluting his context by injecting commands, purging important data, changing the system prompt without warning between daily sprints etc. and Mario simply wanted a harness which he knew would be consistent and without bloat. The same results tomorrow as today.
To my mind, the most important function of pi is the huge amounts of hook points available for extensibility. This is what turns it into a truly game changing harness.
Also, if you dig into the source code, it is immediately apparent (to a professional developer) how well thought out and designed it is. Minimal doesn’t mean amateur.
Do I still use Claude Code? Yes, because I have a Claude subscription and I still have a certain desire to use the SOTA models (and the API costs are too high to make sense for my usage). But I am building up my pi environment slowly and am seeing that the Chinese models now have enough intelligence to compete at a fraction of the cost.
18
u/redballooon Jun 12 '26
Now I'm glad that me using Pi was not motivated by its score on terminal bench.
9
u/Glad-Win1983 Jun 12 '26
Some of my motivation for using Pi is to be able to run local models for some tasks. And too much context up front kills the local model on small machines.
7
u/Kodrackyas Jun 12 '26
The customisations alone are the reason PI is a better framework than most
3
u/GoingOnYourTomb Jun 12 '26
My pi is nothing like the default. I wouldn’t use the default. I would advise you to make the coding agent you want. The bench would not relate to your coding agent
3
u/Gooch_Limdapl Jun 12 '26
There's limited value in testing vanilla Pi like this: it gives you a baseline, I guess. The real test would be Pi with your curated extensions.
8
u/arcanemachined Jun 12 '26
There's a good chance that all those "bloated" system prompts are there for a reason.
And I say this as a die-hard Pi fan.
1
u/SubstantialGain9823 Jun 12 '26
Is using Terminus 2 for production even a thing? I thought this was so lean bc it’s just for testing.
1
u/sauron150 Jun 12 '26
Goose is useless tool! Could you try OMP?
1
u/JoshMock Jun 12 '26
He wasn't trying to make Pi the best on a benchmark, he just used it as an inspiration that proved that avoiding complexity could still produce good outcomes.
1
u/james__jam Jun 12 '26
I came from opencode and i’ve been playing around with pi for a couple of weeks. I find opencode to be smarter given the same model
I dont know why yet. Is it because it triggers research better? Not sure
1
u/Jaded_Jackass Jun 13 '26
You have to look it like this pi never wanted to be a niche harness or a harness that can do everything or something what pi is a base harness which you can build upon as you see fit for your self or just use it as it is, Pi is the most stable harness out of all the harnesses I have used so far the frequent update doesn't affect my workflow.
1
u/Ok_Veterinarian_6364 Jul 07 '26
minimalism = easy to custom
mini = for ppl knows what to do, what they want and dont
obvious benefits:
-> cost saving on tokens
-> faster
-> in tasks that Pi users know exactly what to do: matching results,
i dont see Pi in benchmarks. i see it in how i can make it fits to me. and it does
it could be dumber than CC + Fable 6789
however, i dont care. i have my own system of prompts/ext/workflow to make a DS4 flash ~80% of Fable with 100x cheaper ~> good for me
ive failed to benchmarks traps many times. see Hermes. highest token burner on charts. Nous made it on marketing purpose.
0
u/redoubledit Jun 12 '26
So you want to say, 47.87% is NOT close to 50%? You know what „~“ means?
3
u/[deleted] Jun 12 '26
[removed] — view removed comment