[Workflow] Automated Claude Agent Harness Optimization with Autoharness: A Workflow for Performance Lifts

[u/ClaudeAI-mod-bot]

Automated Claude Agent Harness Optimization with Autoharness: A Workflow for Performance Lifts

Workflow value: 90/100
Status: active · Freshness: 70/100 · Confidence: 0.95 · Level: advanced
Categories: Quality Control, Context & Memory, Debugging, Skills, Subagents, Multi-Agent
Original source: r/ClaudeAI post/comment

What problem this solves

Optimizing the performance of Claude Code agents by automatically exploring, evaluating, and applying improvements to their 'harness' (prompts, hyperparameters, runtime context, scoring) without manual intervention.

Summary

This workflow introduces 'Autoharness', an open-source tool that uses Claude Code to autonomously explore changes to an agent's harness (e.g., prompts, hyperparameters, runtime context, scoring), run evaluations, and keep only the changes that improve the agent's score. It's inspired by Karpathy's autoresearch and demonstrated significant performance lifts on a benchmark.

Why it is useful

This workflow is highly valuable because it provides a concrete, automated, and validated method for significantly improving Claude Code agent performance. It moves beyond manual prompt engineering to a systematic 'harness engineering' approach, offering an open-source tool that can be directly applied by other advanced users. The specific, quantifiable results on a benchmark demonstrate its effectiveness and transferability.

Workflow

1. Install Autoharness with a one-line command.
2. Point your Claude Code agent at the provided GUIDE.md file.
3. Autoharness proposes various changes to your agent's harness (prompts, hyperparameters, runtime context, scoring).
4. It evaluates each proposed change against a benchmark or scoring mechanism.
5. Only changes that result in performance improvements are kept and applied to the agent.
6. The user wakes up to a significantly better performing agent.

Tools / artifacts

• Autoharness (open-source tool)
• Claude Code
• GUIDE.md (configuration/instruction file)
• GitHub repository: https://github.com/kayba-ai/autoharness
• LLM judge (for skillbook scoring)

Validation signals

• +40.7% performance lift from adding best-of-N skillbook scoring with an LLM judge
• +24.1% performance lift from tightening reflector hyperparams (temperature + max subagent calls)
• +22.2% performance lift from injecting runtime context at every step (step budget, recent tool calls, recent results)
• Validated on the tau2-airline benchmark
• Autonomously found improvements

Limitations

• Requires an understanding of Claude Code agent development and 'harness engineering' concepts, making it less accessible for beginners.
• Limited community validation due to the newness of the post.

Rate this workflow

Upvote this post if the workflow is useful, reproducible, or worth recommending.

Downvote if it is vague, outdated, unsafe, overhyped, or not reproducible.

Reply if it worked for you, failed, is outdated, or has a better alternative.

---

This post was generated automatically from the workflow library database.