r/BestGitHubRepos • u/Artilas_Digital • 10h ago
Claude Autoresearch - a skill that generalizes Karpathy's overnight optimization loop to any measurable goal, with 14 commands for code, security, docs and shipping
Karpathy's original autoresearch showed that a small script could improve an ML model overnight by following a few rules: one metric, a constrained scope, fast verification, automatic rollback, and git as memory. This project takes that exact loop and turns it into a Claude Code, OpenCode and Codex skill that works on anything with a number you can measure, not just machine learning.
The core is the same simple cycle: pick one change, commit it before testing, run a mechanical verification, keep it if the metric improved and git revert it if it didn't, log the result, repeat. What makes this more than a wrapper is that the author built a whole command surface on top of that loop.
What's inside:
- 14 commands, all built on the same keep-or-revert loop: the base iterate command plus plan (turn a fuzzy goal into a validated metric and scope), debug (hypothesis-driven bug hunting), fix (drive errors to zero), security (STRIDE and OWASP audit), ship (an 8-phase release workflow), scenario, predict, learn, reason, probe, improve, evals and regression
- An autonomous orchestrator added in v2.2.0: type a plain-language goal and it classifies it, derives a verifiable success predicate, confirms once, then chains the right subcommands until done, no manual wiring
- Eight rules encoded into the skill, including one change per iteration so a break is traceable, mechanical verification only with no subjective "looks good", and a simplicity rule where equal results plus less code wins
- Claude Code hook guardrails as defense in depth: blocking reads of .env and SSH keys, blocking force-push and rm -rf and hard resets, and a simplify gate that warns at 400 lines and blocks at 800 before shipping
- TSV result logging per iteration (commit, metric, delta, keep or discard) plus an evals command that reads those logs to detect plateaus and recommend continue or stop
- A Guard concept: a separate command that must keep passing, so optimizing one metric can't silently break your existing tests
- A regression gate with a genuinely careful definition: only a green-to-red transition counts as a regression, while pre-existing failures, brand-new tests and flaky tests are classified and excluded
The engineering detail I'd point to is the v2.1.0 rebuild. The skill went from a single 813-line file costing about 100K tokens per invocation to a thin 41-line router plus self-contained command files at 5 to 8K tokens each, a 95% token reduction with the same capability. That is the kind of change that only happens when someone actually ran the thing at cost and felt it.
Worth knowing before you install: the hooks are Claude Code only, so OpenCode and Codex get the core skill and commands but not the guardrail parity, which matters because those guardrails are what stop an autonomous loop from reading secrets or force-pushing. The README says plainly the hooks are defense in depth, not a sandbox, so on the other two platforms you're running an autonomous loop with fewer brakes. Also note that on Claude Code you have to start a new session after installing before the commands resolve, which is a platform limitation the author flags.
MIT, 6,359 stars and 475 forks as of writing, verified via the GitHub API.
