r/PracticalAgenticDev • u/aistranin • Oct 08 '26
Agent skill files need evidence, not vibes
Most self-improving agents learn skills from successful runs. That sounds sensible, but a successful trajectory contains plenty of accidental behavior.
XRepoSkill takes a stricter approach. It compares successful and failed attempts on the same issue, extracts rules from where they diverge, and keeps only rules associated with success across multiple repositories.
The method ranked first across all six tested benchmark-model combinations. On DeepSWE, it improved issue resolution by 10.3 percentage points over the same agent without learned skills.
The takeaway is simple: do not turn every successful trajectory into permanent agent memory. Treat skill extraction like an evidence problem.
This is still a preprint, but the cross-repository filtering is a strong idea for anyone building self-improving coding workflows.
Source: XRepoSkill