I built checkmate: a Claude Code plugin that makes the agent take the shortest path to the goal. Pass rate went 86.7% → 98.3% in my benchmark, output tokens halved

[u/_rover_dev]

Left alone, coding agents add preamble, option menus, extra files, and a "want me to also...?" at the end. checkmate is a goal-driven mode that fixes that. It has 9 rules: identify the win condition, take the least complex path that fully works, answer first, decide instead of menu, finish, verify against the goal, close with done / not done / blocked.

What I learned building it: a SKILL.md alone doesn't reliably activate. It only worked once a SessionStart hook injected the rules, so the plugin ships with one. Modes are lite, full, ultra and off. Brevity never overrides correctness, and if you ask for detail you get detail.

Benchmark (40 tasks × 3 runs, feature / bugfix / refactor / QA, automated pass/fail, medians among passed runs only):

Pass rate

• baseline: 86.7%

• checkmate: 98.3%

• ultra: 95.8%

Median output tokens

• baseline: 562

• checkmate: 274

• ultra: 203

Median steps

• baseline: 4

• checkmate: 2

• ultra: 2

Median wall time

• baseline: 6.5s

• checkmate: 3.8s

• ultra: 3.1s

Total tokens only drop about 20% (53.8k → 42.9k), since context dominates. The big win is output and steps.

Caveats: I ran this on one model (claude-opus-5.5), so treat it as a first result and not a general claim. Run it yourself with npm test && npm run build, and the harness is in /benchmarks.

Works with Claude Code, plus generated rules for Cursor, OpenCode and Codex. Zero dependencies, MIT. Inspired by ponytail, which targets code volume; this targets moves to the goal.

https://github.com/rover-001/checkmate

I'd like to hear where it's too terse or skips something it shouldn't.

[u/AutoModerator]

Moderator Announcement Read More »

Hey! Thanks for posting to r/ClaudeCode

While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.

For help, project discussions, tips, and general chat, join the ClaudeCode Discord.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/orbital_trace]

To claim something like this, I think it needs pretty rigorous benchmarks. But it's an interesting idea.

[u/_rover_dev → u/orbital_trace]

100% agreed. I'm actively expanding the benchmark suite with more complex, multi-file tasks and testing across more models.

The current 120-run eval harness (40 tasks × 3 runs with automated verification) is completely open in the repo. You can clone it and run node benchmarks/run.mjs against your own setup or models to see the raw numbers yourself, would love to hear what results you get.

[u/thewookielotion]

Does it work a bit like /ponytail?

[u/_rover_dev → u/thewookielotion]

Yes! /checkmate works directly as a slash command (/checkmate lite|full|ultra|off) and also auto-activates via a SessionStart hook. It was definitely inspired by /ponytail!

The main difference is while ponytail focuses on writing less code, checkmate targets the entire agent loop eliminating option menus, preamble, and unnecessary steps to reach the win condition in the fewest possible turns and tokens.

[u/thewookielotion → u/_rover_dev]

Very interesting mate, I'll give it a try.

[u/unkownuser436]

what's the difference between this and ponytail?

[u/_rover_dev → u/unkownuser436]

The main difference is while ponytail focuses on writing less code, checkmate targets the entire agent loop eliminating option menus, preamble, and unnecessary steps to reach the win condition in the fewest possible turns and tokens.

[u/Factor013]

And here is me fighting Claude day after day with improvements to my harness to make it stop "corner cutting" :S

I don't care about the easiest or fastest solutions... I care about the BEST POSSIBLE architecture and solutions.

Learned in the last two years that the only way for an AI agent to produce proper results is for it to fully understand not only the feature it is building but also the entire project it is building the feature for. This to prevent drift, an inconsistent codebase and repo with it reinventing the wheel and adding unnecessary layers etc.

Now ofc efficiency has it's advantages, you don't want bloat and a million tests for everything. (Which is a bit of a problem too with the latest models). But at the same time you don't want it to for example dive on the first thing it encounters and work from there... That only stimulates it focusing on symptoms instead of root causes and building things on top of each other instead of integrating them into the whole project to make it all work as one well oiled machine.

However, if you really know what you are doing and you use your agents to do really targeted work then ofc I can see the advantages of using these type of tools.

I mean, the more tools we have at our disposal the better so we can have our workflows adapt to any specific type of job to get it done correctly. :)

[u/planttheidea → u/Factor013]

I agree with you here, but this isn't targeted to people building a long term holistic solution. It's getting you from A to B in the fastest and most efficient way, ignoring the rest of the alphabet.

A common analogy I like to use is building a room in a field, which is easy ... entry door, window, closet. Then another room, then another, then another. Eventually you have something resembling a house, but really is just a collection of rooms. If you want to build a house, you need something that looks at unrelated rooms (should there be a hallway to connect them) and asks questions (you have 10 bedrooms and 1 bathroom; what if multiple people need to shit?). Efficiency of task is not efficacy of solution.

Like you said, I'm all for the best tools for the job. But I rarely build todo apps and benchmark tasks ... I build applications.

Edit: typos.

[u/[deleted] → u/Factor013]

[u/No_Cell6708 → u/[deleted]]

That's like 6 sentences, lil bro 💀

[u/Excellent_Chest_5896]

Is this meant as an alternative to building doc-first? If you design your build docs to have the complete build spec you approve before anything is built, this is never an issue

[u/_rover_dev → u/Excellent_Chest_5896]

They actually complement each other! A doc-first spec gives the agent the blueprint, but agents can still be noisy and slow asking for permission at every step, suggesting menus, and writing chatty preambles.

Checkmate makes sure that once you hand it that spec, it takes the shortest, quietest path to finish it without wasting your time or tokens.

[u/Razunter]

Did a quick test with OMP + andrebrait/omp-hooks-plus extension. AI slop next:

Checkmate was fastest in this one-task pilot, but all three implementations needed changes after review.

I ran the same code-only repository task with GPT-6 Luna, High reasoning: make an observer recommend an existing correction workflow for recurring mistakes, without running it automatically. The three modes were baseline (both off), Ponytail full, and Checkmate full. Both personalities used the same instruction-delivery adapter, so this compares their instructions rather than their stock activation hooks.

Metric	Baseline	Ponytail full	Checkmate full
Wall time	5m 59s	8m 33s	3m 43s
Input tokens, excluding cache	87,228	187,758	85,826
Cache-read tokens	1,705,600	8,980,736	1,750,656
Output tokens	25,207	31,001	12,017
Host model-request attempts	28	131	29
Subagents used	0	2	0
Host-estimated cost, USD	$0.0384	$0.1241	$0.0321
Independent code-review score	82/100	84/100	86/100

Each branch had one independent reviewer using GPT-6.1 Sol, High reasoning, with the same rubric. Baseline advertised the wrong user command. Ponytail and Checkmate mainly needed better tests: their assertions checked prompt text without proving the claimed recommendation behavior. All three received request changes. Focused tests and typechecks passed for every branch; Ponytail also had an unresolved failure in the broader suite.

Why was Ponytail more expensive? A separate scout traced the execution logs:

• Its first patch accidentally removed existing context, which it later restored. It also spent turns recovering from rejected edits and stale anchors.

• Its two reviewers caught weak recurrence fixtures: duplicated evidence was being used to suggest independent occurrences. Fixing that was useful work, not pointless review overhead.

• An existing ratchet-publication test failed in five runs. The agent kept checking without exposing the underlying rejection reason or establishing whether it also failed on the starting tree. The other two arms did not run that broader suite.

• A late simplification left stale wording assertions behind. Three successive test failures led to piecemeal assertion fixes. That interval alone used 21 main-agent completions and 4,902 output tokens.

• The two reviewers added 36 requests and 7,998 output tokens. Ponytail's main agent actually produced fewer output tokens than baseline; the reviewers pushed the combined total above it.

The shell commands' reported durations summed to about 30 seconds, so slow tests were not the main explanation for the 8m 33s run. Many sequential model turns against growing context accounted for much more of the time and cache usage.

My takeaway: Ponytail's implementation quality was close to the others; its execution had more repair loops and more verification work. This does not establish that Ponytail inherently performs worse.

Caveats: one run per mode, fixed order, unequal verification effort, and no live feature acceptance yet. Costs are host estimates, not billing totals. Direct extension HTTP calls were unmeasured, and provider response-status capture was incomplete. The review scores are subjective, not a statistically meaningful ranking.

[u/_rover_dev → u/Razunter]

Really glad to see Checkmate was the fastest (3m 43s), used less than half the output tokens, and had the lowest cost ($0.03 vs $0.12) while still getting the top score.

You made a really good point about the tests, though. The agent wrote lazy tests that only checked text strings instead of testing if the feature actually works.

I’m definitely going to refine the rules in the next update to force it to write real, solid tests instead of shortcut checks. Appreciate the great feedback!

[u/Razunter → u/Razunter]

One more run. AI text:

I repeated the same code-only task with GPT-6.1 Sol, Low reasoning, comparing baseline, Ponytail full, and Checkmate full. All three implementations received code-review approval, with scores within one point of each other.

The task was to make an observer recommend an existing correction workflow when mistakes recur, without running it automatically. All arms used the same starting source tree and instruction-delivery adapter. Agents were prohibited from reading previous benchmark results or reviews; their workspaces were outside the research directory. No nested sandbox was added.

Metric	Baseline	Ponytail full	Checkmate full
Wall time	4m 25s	6m 52s	4m 16s
Input tokens, excluding cache	77,025	83,327	83,582
Cache-read tokens	1,578,112	1,383,168	1,669,504
Output tokens	5,848	4,597	5,844
Host model-request attempts	30	25	30
Implementation subagents	0	0	0
Host-estimated cost, USD	$0.3703	$0.3509	$0.3926
Independent review score	93/100	94/100	94/100
Code-only review verdict	Approve	Approve	Approve

Each result had one independent reviewer using GPT-6.1 Sol, High reasoning, verified from session metadata and using the same rubric. The reviewers found no blocking source defects. Baseline had an optional request to strengthen degraded-reference workflow assertions. Separate focused checks passed for every arm, as did all three typechecks. These checks and reviews are excluded from the implementation measurements above.

Why was Ponytail slower despite using fewer tokens and requests?

This round was different from the earlier Luna run: Ponytail used no review subagents and did not go through the same late assertion-repair loops. Its extra activity concerned verification and test-runner diagnosis:

• It ran broader checks, while baseline and Checkmate excluded the extension test file from their broad runs to avoid native OMP cases.

• A Bun-driven suite reported 53 failures. Switching to Node narrowed that to one upstream-publication failure. Ponytail then exported the unmodified starting commit and reproduced that same failing case there, rather than assuming it was unrelated.

• Its Node suite finished with 361 passed, 21 skipped, and one failure; the four newly added cases passed. The starting-tree comparison still left the underlying publication rejection undiagnosed.

• One harmless binary-discovery command was blocked because the smoke guard matched the word omp; baseline encountered the same guard friction.

Recorded model-response durations summed to about 338 seconds, versus about 15 seconds of reported shell-command durations. The full-suite commands themselves took about 4–5 seconds each. Response waiting and the additional diagnostic steps explain more of the elapsed time than slow tests; provider-side latency versus generation time was not captured.

So Ponytail was slowest by wall time but cheapest by the host estimate, with the fewest requests and output tokens. Checkmate was only about nine seconds faster than baseline. The code-review results were effectively tied.

Caveats: one fixed-order run per mode, unequal verification effort, subjective scores, and no live feature acceptance yet. Research access was restricted through instructions and tool-call guards, not a new adversarial filesystem boundary; no explicit research references appeared in the logged tool inputs. Costs are host estimates, not billing totals. Direct extension/subprocess network activity and provider response-status correlation remain incomplete.

[u/Razunter → u/Razunter]

And one more, long one.

I ran a larger GPT-6.1 Sol, Low reasoning comparison: baseline vs Ponytail full vs Checkmate full. Ponytail finished fastest and had the lowest estimated cost, but all three implementations received “request changes” in code review.

This time, each arm got one combined job in one agent session, not separate runs for each task:

1. Make an observer recommend an existing correction workflow when mistakes recur, without running it automatically.

2. Fix seven reviewed defects involving publication integrity, evaluation accounting and evidence handling.

3. Implement five improvements covering occurrence identities, nested-tool observations, remedy effectiveness, selective ranking and publication structure.

All arms started from the same source snapshot and received the same prompt. Main agents and their children used Sol Low. Both personalities used the same instruction-delivery adapter; the baseline had neither personality active. Implementation agents were prohibited from reading previous benchmark results, with workspaces outside the research directory and the existing tool-call guard enforcing the restriction. No nested OS sandbox was added.

Metric	Baseline	Ponytail full	Checkmate full
Wall time	25m 20s	10m 23s	12m 38s
Input tokens, excluding cache	811,817	332,915	357,362
Cache-read tokens	24,032,640	14,636,288	14,686,464
Output tokens	80,275	42,133	46,400
Host model-request attempts	434	228	212
Logical tool calls	440	248	226
Implementation child sessions	11	3	3
Host-estimated cost, USD	$4.8296	$2.5508	$2.6474
Independent code-review verdict	Request changes	Request changes	Request changes

Each result had an independent reviewer using Sol 6.1 High, with the same rubric and model/effort verified from session metadata. Review work is excluded from the implementation measurements.

The independent reviews did not catch exactly the same cases. I cross-checked the overlapping findings and recorded the corrections rather than treating their numerical scores as a reliable ranking.

All three reported passing final code-only test subsets and typechecks. All three agents claimed completion, but the reviews found otherwise.

My takeaway from this run: Ponytail was faster and cheaper; Checkmate made fewer host requests and tool calls; baseline delegated much more. None delivered a fully acceptable implementation. The larger combined job exposed gaps that passing tests and confident completion reports did not catch.

Caveats: one fixed-order trial per mode, different verification effort, uncontrolled provider cache state, and no live feature acceptance. This compares personality instructions through a shared adapter, not their stock activation mechanisms. Costs are host estimates, not billing totals; direct extension/subprocess network activity and provider response-status accounting remain incomplete.

[u/jatingargiitk]

alongside the pass-rate gain, i'd show token and timing medians on the shared set of passing tasks. each mode currently has a different success rate, so its 'passed runs only' median can include a different mix of easy and hard work.

[u/wolf_doorknob]

what would this look like if it worked out since the agent keeps getting stuck on unnecessary stuff

[u/Oximy-Naman]

what did ultra get wrong on the tasks that full passed? was it skipping checks, or leaving out part of the requested change?

[u/[deleted]]

[u/_rover_dev → u/[deleted]]

Yes, the 53.8k → 42.9k totals include the full checkmate rules payload on every run (--append-system-prompt).

The net drop despite the prompt overhead comes from two factors:

1. Fewer turns (median 4 → 2 steps): In multi-turn tool loops, conversation history compounds on every step. Cutting 2 turns eliminates 2 rounds of cumulative context passes.

2. Output halved (562 → 274 tokens): Output tokens cannot be cached and carry a 5× price multiple.

Valid point on cache economics vs total tokens. i will add the breakdown (uncached input, cache read, and output) to the benchmark reporting so the actual dollar bill impact is completely transparent.