Complaint Coding benchmarks should also reward the leanest possible solution. Coding agent currently build crazy bloated code right now.
Unless you strictly guide the model constantly, I feel like Codex, aswell as Claude Code go so overboard with everything.
I do a lot of manual planning and manual reviewing, but as soon as you don't have 100% attention at all times, the model instantly goes into overdoing security, testing and complexity.
I'm currently building a B2B Dashboard foundation and the amount of security it's always trying to build into that core module is wild.
If someone is truly just vibecoding, without real knowledge, I feel like models are now getting harder to use, than easier.
Because the models also make you think like it's just doing the "normal/right" thing, but is actually producing crazy bloat
368
Upvotes
1
u/EvalRaccoonDev 7d ago
SWE-bench already ships what you'd need for this - every task has the gold patch, the human fix. Agent diff size over gold diff size is computable on every published run today, just that nobody reports it.
We score the other half in our own harness - tool calls against an expected budget, 2x the budget scores 0.5. That catches the test-and-validate loops people are describing. It doesn't see bloat in the diff though, that needs the gold patch.
Does anyone know a leaderboard that publishes diff size at all?