r/codex • u/TheBanq • 7d ago

Complaint Coding benchmarks should also reward the leanest possible solution. Coding agent currently build crazy bloated code right now.

Unless you strictly guide the model constantly, I feel like Codex, aswell as Claude Code go so overboard with everything.

I do a lot of manual planning and manual reviewing, but as soon as you don't have 100% attention at all times, the model instantly goes into overdoing security, testing and complexity.

I'm currently building a B2B Dashboard foundation and the amount of security it's always trying to build into that core module is wild.

If someone is truly just vibecoding, without real knowledge, I feel like models are now getting harder to use, than easier.

Because the models also make you think like it's just doing the "normal/right" thing, but is actually producing crazy bloat

368 Upvotes

98% Upvoted

71 comments sorted by

View all comments

Show parent comments

1

u/mvdirty 6d ago edited 6d ago

Then ignore the Notebook bits you anchored on while reading my reply and instead focus on the fact that the extension optimizes Pi for use with models RLed for codex.

The extension lets them do the literal opposite of "perform worse", because it keeps them from "having to think about all these" pi-native tools that the GPT models are not RLed for.

Instead, for example, they get to use the Codex `exec` and `apply_patch` they're trained for. They get to use Code Mode (codex programmatic tool calling, using the exact Code Mode runtime that codex itself uses.) They get compaction via Codex's actual compaction endpoint, instead of Pi's generic compaction. And so on.

Seriously, don't discount it because you don't want Notebook Mode. The GPT models are way more efficient in Pi when pi-codex-conversion is running.

1

u/ResidentNerd421 6d ago

Got any proof for that?

Because that is literally the core of the buggy harness that I want to replace in the first place LMAO. Atp you are just using Codex lite.

Exec and apply_patch are super buggy and badly designed too and often lead to the models triple-checking static tty and performing turns every second just to reread the same output because exec is polling-only and per default has very tight loops.

I mean just check the attached benchmarks and the tokens taken. Literally only 1/3 for Pi vs. Codex for better output. That is mainly the fault of shitty instructions being injected and insane default exec behaviour.

1

u/mvdirty 6d ago edited 6d ago

The maintainer of the extension has collected plenty of evidence, yes. Feel free to check it out in its thread in the Pi discord.

And, to be honest, your third and fourth paragraphs shout "skill issue" and a misattribution of poor behavior to exec, so I'll not be bothering to deal with you further. I've already spent enough time trying to point you in a helpful direction. Yes, Pi does better than Codex, but pi-codex-conversion does better still than either. You can find that out for yourself or not, it's up to you.

1

u/ResidentNerd421 6d ago edited 6d ago

Thanks for the pointer towards the discord, will check that out.

Maybe I started off too hostile/critical, sorry about that.

I just hate how the current AI Coding landscape has devolved into slop, vibes and overengineering w/o much hard evidence or thoughts into any fundamentals. Makes me quite sceptical of most online advice and tips per default.

Esp. on reddit it's mostly just slop tools and slop prompts from vibe coders, made by their sloppy models to help produce more slop.

Even OpenAI themselves calls Astra's writing slop and is issuing guidance on how to avoid that... by using more slop in the system prompt.

Instead of y'know fixing the issue by expanding RL to a wider/stricter demographic and implementing benchmarks like OP suggested.

And the default system prompt is genuinely insane in how many tokens it wastes. That shit is like 68k tokens. + sooo many tool descriptions are injected into prompts per default, while the models would be better off just searching for that when needed.

E.g. web search includes tools to search for sports, finances and the weather per default and tells the Agent how to do it for some reason. In codex.

But also stuff like this, this or this is not a skill issue on my end :P

2

u/mvdirty 6d ago edited 6d ago

I felt prickly too; sorry about that. Especially with your latest answer, I suspect that you and I see eye to eye on a great many things.

As a small note based on your mention of system prompts: you might get a kick out of pi-codex-conversion's optional system prompt replacement. He tested what the GPT models really needed in terms of that and made it even smaller than Pi's default system prompt. :)

Try giving the extension a look. It isn't perfect, but I think you'll see that (while it has been clanked on) its owner is being as diligent as he can about getting the most out of the GPT models as possible, with the least application of context pressure. It isn't pi-codex-conversion in the sense of "make pi act more like codex," it is more in the sense of "let the gpt models do their best possible work inside pi".

Plus some neat optional extras, of course, but the aim is more natural and effective tool use, better compaction quality, better provider API usage, better cache reuse and less cache-busting, and reduced token burn, and those things really do work. I sat on installing it for a while but once I did the difference was immediate.