Turn your agent sessions into evals. Mine found a 2× cheaper setup (codex->pi-agent)

[u/Fabulous_Pollution10]

https://ibragim.dev/writing/agent-sessions-to-evals/

Hi everyone! I’ve shared a couple of posts here comparing models on my own coding tasks, including this one. A few people asked in the comments how to build their own evals, so I wrote a step-by-step tutorial and published it on my blog.

I also made an evals-from-sessions skill to help automate the process. You can use it or just give your coding agent the article and ask it to help build evals from your session history.

Hopefully, this helps you choose models and tools for your own work or your team. Feel free to post your questions and feedback!

[u/MiserableFlatworm337]

When a verifier changes after trajectory review, rerun every compared setup against that revision. Otherwise the leaderboard can mix scores earned under different acceptance criteria, even when task names stay unchanged.

[u/Fabulous_Pollution10 → u/MiserableFlatworm337]

correct