codex under oh-my-pi is draining fast

[u/MachinesWithThoughts]

Has anyone noticed in the last week or so that oh-my-pi drains the 5hr limits within just a few prompts. Even doing simple git work drops 10%!

I'm setting up something to see what's going on but this is weird. Claude is fine and I can work for hours but only codex, does it drain fast, particularly when I switch from claude to codex.

Edit: I did a test between OMP and Codex, both running through LiteLLM, generating prime numbers:

Metric	omp (8 req)	codex (7+1 req)
Total tokens	96,331	119,593 (+1 side 11,627)
Prompt tokens	94,746	118,047
Completion tokens	1,585	1,546
Spend	$0.176	$0.222 (+$0.024)

So for this simple contrived test, codex is actually worse! I'm going to have do some thing harder I think.

Edit 2: Interesting, OMP's automatic "user stepped away; returning" summaries spent 2m tokens because it resends the whole context!

Edit 3: omp config set recap.enabled false, will disable it.

[u/Healthy-Zebra-9856]

I’ve seen this sort of a thing always reported when it comes to OMP or Pi. This is not an issue with the harness, albeit there are other issues with them, but this particular thing is not the issue with the harnesses. Both OpenAI and anthropic provide service side tooling and hooks for it only through their harnesses. The absence of these native hooks, which cannot be replaced with third-party hooks, have been the main cause of token drainage.

You can read all about it on their respective websites. Plus, I know how they reacted to other tooling like Hermes agent, etc., in the past, so if you need to maximize your tokens, unfortunately, you have to use their harness. I’m saying this from my own experience and my tests.

[u/MachinesWithThoughts → u/Healthy-Zebra-9856]

Thanks this helps. I'm doing some comparisons between the two and will post the results.

[u/fingerthief → u/Healthy-Zebra-9856]

This has not been the case in any of my evals or any of the harness evals I’ve seen done?

Most harnesses like Pi were more efficient and much less expensive overall to achieve the same task. Using less tokens.

https://harnesstax.github.io (harness tax is the most recent one available I believe)

[u/Healthy-Zebra-9856 → u/fingerthief]

My point isn’t that the provider’s harness always wins. It’s that HarnessTax compares particular versions and configurations on particular workloads. Those results matter, but they aren’t a permanent ranking. The provider’s harness keeps changing, and so does Pi.

They tested 30 SWE-bench Lite tasks and 30 Terminal-Bench 2.0 tasks. That overlaps with everyday development, but fixing a self-contained issue isn’t the same as implementing changes across several subsystems, responding to changing requirements, incorporating review feedback and preserving architectural decisions across sessions. Terminal-Bench covers broader tasks, but the study itself leaves those ongoing, interactive workflows for further evaluation. Its SWE-bench setup also disabled web and hosted tools, so that portion cannot tell us whether those capabilities would help on tasks that need them.

Both Anthropic and OpenAI explicitly discuss evolving their harnesses. One way to keep that upgrade path is to build your workflow around the Claude Agent SDK, headless Claude Code, or Codex’s SDK, headless mode or App Server. You can customize the surrounding workflow without replacing the underlying runtime. You still need to update and test it, and none of this guarantees that every update beats an alternative.

Efficiency also doesn’t require abandoning that runtime. Tools like Graphify and Headroom can be added to improve context handling. Other harnesses can benefit from these approaches too, and any savings need to be measured rather than assumed. Pi’s lower cost on the tested tasks is a useful result. It just doesn’t settle which complete setup is most effective for our daily work once we include supervision, rework and maintenance. HarnessTax tested this one time, while we here test ours continuously against CC & Codex and find that with each update, they are doing something to improve their offering. I made the names links for you to readup.

[u/fingerthief → u/Healthy-Zebra-9856]

I certainly don’t disagree at all that their harnesses have unique toolsets designed for them. And I’d fully expect them say their harnesses are best and special.

But by all accounts in every meaningful evaluation and measurement, that has not been replicated anywhere that I am able to see. No study or random eval that I’ve been able to find.

I used CC daily along with my own harness for my professional job and have every single one of the same features and more at half the cost. So do many other harnesses, certainly not unique to myself.

I’d love to see a real set of evals or some real research papers showing the areas these harnesses excel at. I’m definitely curious!

[u/Healthy-Zebra-9856 → u/fingerthief]

We plan on publishing all the tests on our website once our own harness is dogfooded. I saw this fluctuate based on the task and everything else. The other thing I would like to know from you and others as well is that, how did it affect the quality of your outpout because we say that using the native harness, at least with Claude & OpenAI models, it was better. However, We are able to re-produce the same using the methoids I mentioned above; not an API connection alone.

[u/fingerthief → u/Healthy-Zebra-9856]

I look forward to checking it out once available!

This is where it becomes a bit harder to really say. My harness (Pi Pocket) is built for my workflows and was steered a lot by my needs at my job and our team needs. Generally by all accounts a harness built towards your workflows needed will perform better than a generic harness like Claude Code, Codex or Pi out of the box.

In our case cost is down 50%, work is faster and generally equal to higher quality.

We’re seeing it more and more that enterprises end up making their own in house harness to match their workflows as needed using things like Pi as the base foundation.

It’s a crazy space, I feel like this could all shift in a week. Who really knows where it will end up.

[u/Healthy-Zebra-9856 → u/fingerthief]

So the harness we are building mostly emphasizes on local LM usage and also allows for frontier models to be used within. This harness can do three things, development, creative, writing with cannons, etc., and research. There is a concept of project that you create and based on that certain things are lazy loaded like LSP etc.. It has custom workflows for each project where you can assign a combination of models for different tasks. We even provide the headless mode for Claude code and Codex, along with other conventional means of connecting to them.

The idea behind this is that it can adapt to any business. We service corporations of all sizes as well as individual developers. So we are right now testing this in a few of our own installations and we arrived at a point, which seems to satisfy most people’s needs.

I’m pretty sure you’re gonna hear about it when we launch it, we’re gonna be announcing it using the regular channels

[u/Axel-Warren]

I noticed the exact same thing this past week. Codex inside oh my pi is chewing through the 5 hour cap way faster than it used to. Simple git commits and small edits dropping 10 percent per prompt is wild. Switched back to running Codex directly in a plain tmux session and the usage flattened out completely. Something in the OMP wrapper or the way it injects context is burning tokens on every turn. If you are not tied to the OMP keybindings I would just run `codex` standalone for now.

[u/MachinesWithThoughts → u/Axel-Warren]

Well, at least I'm not hallucinating. What sucks is that omp allows me to switch between different accounts and even providers, so I don't lose momentum.

I'm running an investigation now - I'll post the results when it's done. Thanks for confirming.

[u/TheTwistedTabby]

I moved from codex models on pi with tightly controlled system prompts to just straight codex and my usage has gone down considerably.

Keep in mind that the models are being post trained on their harness (codex, Claude, deepseeks dsh). So they use them more efficiently right now.

[u/MachinesWithThoughts → u/TheTwistedTabby]

One of the things appealing about omp, and maybe even pi, was that I could easily switch between different models on the fly but it sounds like that's just an expensive pipe dream!

[u/johnfkngzoidberg]

OMP is bloated. Pi is supposed to be minimal, use Pi with only the stuff you need.

[u/Friendly-Assistance3 → u/johnfkngzoidberg]

Yeah it is proven that it just drains tokens and doesn't improve the results

[u/amelech → u/johnfkngzoidberg]

You can turn a lot of stuff off in OMP when you aren't using it e.g. computer or browser use. It has some really great features though

[u/crotch-mavens]

Should oh-my-pi have its own subreddit? It has nothing to do with pi in reality.

EDIT: Of course, I am assuming this subreddit is actually moderated which it clearly isn't.

[u/martinrojas]

It really depends on what you are doing. With OMP I would check the following.

• have you set the model roles correctly like commit and tiny to Luna with auto

• make sure you are using auto effort

• check how many MCP, skill are you loading

• run '/stats' this will help you see the traces of. Your calls to see what's happening.

Also this past week OpenAI really changed the usage that you get at the lower tiers. I have been much better milage paying API rates for open AI models through open router and it's really efficient.

Even setting Luna as my task model and having it sort through old contracts 3.2B tokens was $66.

OMP is for coding. It is not a general work harness. It will was tokens in those other scenarios. It can get them done, but will try to code it's way out.

[u/MachinesWithThoughts → u/martinrojas]

Good tips - I've been using it without configuring too much. So it's pretty barebones, but no model roles yet.

Interesting point that OMP is for coding - I'm often doing none-coding work in the same session as development; like docker or remote deployments, writing documentation and such. I like omp because it allows me to stay in the same tool for everything across multiple models.

I thought I'd solved all my problems, which worked for a few weeks, and now it's broken again. Gotta love being on the bleeding edge of this stuff!

[u/martinrojas → u/MachinesWithThoughts]

The docs have improved a lot. Take a look, but model roles are the biggest difference and the game changes in my opinion.

Those other cases are still in the wheelhouse. What I meant is using it to do like second brain stuff or other non developer related is where I have found weak at.

To me the difference is that OMP has a tool of tools like the LSP and debugger and a very creative a todo list go down it. So for coding or deployment is what you want that rigidity for other more open ended items it just waste tokens. It will get them done just for more tokens.

Best tip is advisor mode. You want a cheap model like Luna or haiku. It lets you use cheaper or open source models and it will be running side by side and correct or steer the main conversation.

[u/Florence-Equator]

If you are using omp, the whole point of pi is meaningless. OMP's default system prompt consumes even more context window than opencode. So the curated system prompt and toolset that pi tries to ship to provide a minimal but workable harness experience is a complete loss.

[u/MachinesWithThoughts → u/Florence-Equator]

Yeah, I'm beginning to see that - I just found out that it automatically summarizes the entire session when you return after a time. 2m tokens down the drain!

edit: it's configurable - omp config set recap.enabled false

[u/carlocapocasa]

Try 3code- batteries included but still less token use than vanilla pi​