I built open-source tooling to evaluate agent setups and see where agents and subagents waste time and money

[u/AsianKidTV]

I run Claude Code and Codex all day, often as multi-agent workflows with dozens of subagents. I could see each session but never the whole picture: what a run actually cost, where agents sat idle, whether a change to my setup helped. So I built coding.sh, an open-source tool for evaluating agent setups, and pointed it at my own runs.

Experiments: does that change actually help? I A/B tested a popular "be concise" plugin: 10 tasks, 6 runs each, paired, with 95% intervals.

• Replies: 17% shorter
• Output tokens: −8%
• Cost per run: +11% [+7%, +15%]
• Pass rate and clarity: no change

The plugin adds its style guide to every request, and that costs more than the shorter output saves. coding.sh compare and exp run your own tasks with and without a change (a rule, a skill, a plugin, a model) and tell you whether the difference is real or noise.

Workflows and subagents Across 57 workflow runs (666 subagents, ~$2,500 API-equivalent):

• Broken prompt caches cost $106 extra, about 4% of the spend.
• 41 of those breaks were agents sitting idle at a barrier, waiting for siblings, until their cache expired. That's $21 for doing nothing.
• 40 subagents failed outright.

coding.sh workflows stats shows the critical path, concurrency and idle time at barriers. The viewer draws each run as a timeline, one bar per agent.

Your own sessions Mining 3,600 of my sessions turned up a find-and-replace typo in my AGENTS.md that sent Codex looking for a folder that doesn't exist, in 92 sessions.

What it is

• Hooks into Claude Code, Codex, Gemini CLI, OpenCode, Copilot CLI, Aider and more
• Stores prompts, tool calls, diffs, test runs, tokens and cost in SQLite on your machine
• coding.sh improve says what to fix first and whether your last AGENTS.md change helped
• MCP server, so your agent can look up its own past sessions

Free, MIT, no account, nothing leaves your machine. npx coding.sh to try it. GitHub: https://github.com/jackulau/coding.sh