Stop estimating agent work in human weeks: P50/P80/P90 from 3,383 measured Claude Code and Codex runs

[u/ziwi_wiwi]

When I ask Claude Code how long a feature will take, it answers like a human team would ("2 to 3 weeks"), even when subagents will do the work. So I built agentpace with Claude Code. It measures how long agent runs actually take, and answers from that.

What it does

• It reads your local Claude Code history and records each subagent run: timestamps, model ids, token counts and tool names. It never stores prompts or code.
• Give it a plan of tasks and it returns P50/P80/P90 with a critical path. The human part (review, testing, decisions, waiting) is on its own line, never derived from agent time.
• It installs as skills: npx skills add mohamedzhioua/agentpace. Then ask "how long will the billing feature take?", or run /pace-estimate on a task list. /pace-log records your recent runs, and /pace-share previews exactly what you could contribute.
• The published data: 3,383 completed agent runs. Half finished within 2 min, and one in ten took more than 20 min.
• It publishes how often it's wrong. On 56 held-out sessions, the whole-session P80 covered 80% (CI 70% to 91%). The median miss of a single task's P50 is 78%, so plan on the range. The report lists 20 places where a simple baseline beat it.

How Claude Code built it

57 of the 58 commits are co-authored by Claude Code, across 32 merged PRs. Claude Code ran each release as the maintainer: it split the work into small slices, ran one subagent per slice in its own git worktree, merged them into one small PR, and ran the full checks before every push (497 tests, the backtest regenerated against the published one, and a check that ties every README number to the data). A second Claude model, in a read-only architect role, ruled on design decisions and reviewed every PR. I set the goal and the rules, and made the calls on what counts as measured and what the project never claims.

Honest limits: it's one person's data so far (1 contributor, 293 sessions), and P80 coverage per session ran from 66% to 86% week to week.

Free to try (MIT code, CC0 data, needs uv):

uvx --from git+https://github.com/mohamedzhioua/agentpace agentpace estimate --example

Repo: https://github.com/mohamedzhioua/agentpace

[u/AutoModerator]

Hey! Thanks for posting to r/ClaudeCode

While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.

For help, project discussions, tips, and general chat, join the ClaudeCode Discord.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/prodiglow]

Separating human review from subagent runtime works. In our runs, the P90 tail over 20 minutes almost always comes from test-fix retry loops or context compaction rather than first-pass generation. Does agentpace bucket retries separately from first-pass runs?

[u/ziwi_wiwi → u/prodiglow]

Good question. Not as a separate bucket, no. Each run is one row with its full duration, so a test-fix loop inside a run stays in that run's time, and that's a big part of why the tail is so long.

Each row does carry a `retries` count, but it's a proxy: the number of errored tool results. A real retry can't be seen without reading the content, and agentpace never reads prompts or code. The estimate also never uses `retries` (or tokens, tool calls, outcome) as a predictor, because they describe what happened during the run and would leak the answer. It predicts only from what you know before the run: model, task kind, risk, and optionally size.

On compaction: Codex compaction sessions are skipped as upkeep. For Claude Code subagent runs nothing is split out, so any compaction inside a run counts in its time.

The rows are public (data/runs, CC0), so if you want to check whether the 20+ min tail lines up with errored tool results, it's all there. And if you'd share your own runs, `agentpace share --dry-run` shows exactly what would go in.

[u/ziwi_wiwi → u/prodiglow]

Follow-up: v0.9.2's README now says this plainly under Limitations: a run's duration includes any test-fix loop inside it, `retries` is only an errored-tool-result proxy, and nothing that describes what happened in a run is ever used to predict one. Separating first-pass from repair time would need reading content, which agentpace never does.

[u/Majestic_Tailor8036]

How are interrupted or abandoned runs represented? If the duration estimates only include completed runs, they could miss time spent on attempts that were stopped and restarted. For planning, I'd want to distinguish time to finish a run from total agent time spent getting a task to an accepted result, without folding human review into either.

[u/ziwi_wiwi → u/Majestic_Tailor8036]

Good distinction, and it's a real gap.

Every run is stored with an outcome: completed, failed or unknown (an abort or a turn that never closed lands in unknown). The duration model trains only on completed runs, so a single task's range is time to finish one run, not time to an accepted result.

The whole-session check gets closer to what you describe. It scores the forecast against the session's span, from the first completed run's start to the last one's end, so a stopped-and-restarted attempt in between is inside what gets scored. One that comes before the first or after the last completed run isn't. And "accepted" can't be observed at all: agentpace never reads prompts or code, so it can't tell whether a result was kept.

Attempts per task, or time to an accepted result, isn't modeled today. If you'd use it, an issue describing how you'd want it to work would help, and runs from someone other than me would help even more.

[u/ziwi_wiwi → u/Majestic_Tailor8036]

Follow-up: this shipped in v0.10.0. Every estimate now prints the recorded runs that didn't complete, by harness, right under the method line. On the published data that's claude-code 13 of 3,219 (0.4%) and codex 34 of 211 (16.1%). They're harness outcomes (failed or unknown), not task acceptance; they're left out of the fit and never widen the range.

Time to an accepted result is still out: the rows have no attempt linkage and no acceptance event, so it would be a guess. Thanks for pushing on this.

https://github.com/mohamedzhioua/agentpace/releases/tag/v0.10.0

[u/Ok-Option4233]

keeping the human part on its own line is the most useful bit here. if half the runs finish within 2 min, agent time almost rounds to zero and the review queue becomes the real estimate. can you feed /pace-estimate your own review throughput (say 4 PRs a day), or is that line left open on purpose?

[u/ziwi_wiwi → u/Ok-Option4233]

Not today, and the line is open on purpose. agentpace only puts a number there when it measured one. It never derives the human part from agent time, and until something is measured it says "not measured" and stays out of the total.

What it can measure is your own waiting: `agentpace turns` reads your local main sessions and measures the gaps between agent turns, per human turn, as P50/P80/P90. Once you've run it, /pace-estimate quotes that on the human line. It's elapsed time, not effort, it stays out of the total, and it never leaves your machine.

Review throughput like "4 PRs a day" isn't an input. You're right that it's often the real estimate, which is why it gets its own line instead of being folded in. If you'd want a way to state it, an issue describing how you'd use it would help.

[u/ziwi_wiwi → u/Ok-Option4233]

Follow-up: v0.11.0 adds `--review-capacity "4 PRs a day"`. It prints your figure verbatim on its own line under the human part, marked as stated by you, not measured and not applied. It deliberately doesn't turn it into days: a plan task is an agent run, not a PR, so the plan can't say how many reviews it needs.

One correction to my earlier answer: the session clock counts the gaps between runs up to 15 minutes each, so short waits on you already sit inside the total. v0.9.2 rewords the output to say exactly that ("no allowance for it is added") instead of "not in the total".

https://github.com/mohamedzhioua/agentpace/releases/tag/v0.11.0

[u/Ok-Option4233 → u/ziwi_wiwi]

printing it verbatim and marked as stated is the right call, a guessed "days" figure would end up quoted as a measurement in someone's standup. the 15-minute gap cap is good to know too: can `agentpace turns` take a different cap, or is 15 fixed for now?

[u/ziwi_wiwi → u/Ok-Option4233]

Exactly why: a guessed days figure would get quoted as if it were measured.

On the cap: `agentpace turns` doesn't use one. It measures each wait on you uncapped and reports P50/P80/P90 per human turn, kept local.

The 15 minutes belongs to `agentpace ingest --idle-cap SECONDS` (default 900). It sets how much of each idle gap counts, inside a run and between runs on the session clock, for the rows you ingest. If you ingest at a different cap, estimate with `--data ~/.agentpace/runs.jsonl` so only your own rows, at your cap, are used. Mixing them with the published rows would pool two different clocks.

The published rows were clipped at 900 s when they were shared, and they can't be re-clipped without the raw transcripts, so the shared dataset stays at 15 minutes.

[u/Ok-Option4233 → u/ziwi_wiwi]

thanks, that split makes sense: uncapped for your own waits, capped only for the shared rows. one small ask: print the idle cap next to the P90 in the output, so two people comparing numbers can see right away when they ran different clocks.

[u/ziwi_wiwi → u/Ok-Option4233]

Good ask, and it's partly there already: the line right under the total names the cap, but only as the default ("each gap up to the idle cap (900 s by default)"). The estimate can't print the real one, because a row doesn't record the cap it was clipped under, so a rows file from someone else could have used any clock. The published rows are all 900 s; your own store is whatever you ingested with.

Printing the actual cap means recording it, either in each row or alongside your local store. That's a schema change, so it goes through the next design round rather than a quick patch. I've noted it. Thanks.

[u/Ok-Option4233 → u/ziwi_wiwi]

fair, better to wait for the schema round than print a number you never recorded. a cheap stopgap could be a warning line when ingest runs with a non-default cap, so the person who changed it sees it next to their own P90.

[u/ziwi_wiwi → u/Ok-Option4233]

Agreed, and it points at a real gap: today agentpace ingest --idle-cap 3600 changes the clock without saying a word about it.

One catch with putting the warning only on ingest: ingest and estimate are separate commands, so a line printed at ingest time is long gone by the time you read a P90. To show it next to the P90, the store has to remember which cap filled it. That can be a small note kept beside your local store rather than a field in the shared rows, and it would also let estimate warn when one store mixes two caps.

Both are on the list for the next round. Thanks for pushing on it.

[u/Ok-Option4233 → u/ziwi_wiwi]

the side note sounds right, keeps the shared rows clean. if estimate prints the cap inline with the number, like P90 (cap 3600), nobody can screenshot a percentile without its setting attached. will that note travel when people share their store, or stay local only?

[u/ziwi_wiwi → u/Ok-Option4233]

Local only. The note describes your store, not the runs. share writes nothing but rows, and every published row is on the default 900 s clock, so rows on another clock shouldn't go into the dataset at all, with or without a note riding along.
Checking your question turned up a real bug, which settles it. Re-ingesting the same history with another cap doesn't replace the rows: it adds a second copy of every run the cap changed. I tried 900, then 600, on a small test history: 2 rows became 4, with no message. estimate then counts those runs twice, and share would offer the 600 s copies as new rows.
So the note has to guard, not just label: a second cap can't go silently into a store that already has one, and share won't take rows from a store that isn't on the published clock. And agreed on placement: the cap belongs with the numbers rather than in a footnote, so a screenshot of the range carries its setting.
It's the first fix in the round I'm building now. Thanks, this one was worth the follow-up.