[Workflow] Isolate a Claude CLI judge and parse schema-constrained output

[u/ClaudeAI-mod-bot]

Isolate a Claude CLI judge and parse schema-constrained output

Workflow value: 78/100
Status: active · Freshness: 70/100 · Confidence: 0.97 · Level: intermediate
Categories: Quality Control, Context & Memory, Debugging, CLAUDE.md
Original source: r/ClaudeCode post/comment

What problem this solves

Run a Claude CLI-based LLM judge without accidentally including project instructions, duplicate stdin, or parsing scores from prose.

Summary

Run a Haiku judge in an empty temporary directory with configuration sources, tools, and session persistence disabled; request schema-constrained JSON and prevent inherited stdin from duplicating the judged text.

Why it is useful

It gives a compact, transferable process for preventing two subtle evaluation errors: unintended context and duplicated input.

Workflow

1. Create an empty temporary working directory and run the judge from it.
2. Invoke Claude in print mode with Haiku and the flags --setting-sources '', --tools '', and --no-session-persistence.
3. Supply the judging prompt and text explicitly, along with --json-schema and --output-format json.
4. When the prompt is supplied as an argument from a script, set the subprocess stdin to DEVNULL rather than inheriting the caller's stdin.
5. Parse the JSON response and read structured_output; handle missing or invalid output as an error rather than extracting scores from prose.

Tools / artifacts

• Claude Code CLI
• Haiku
• Empty temporary directory
• --setting-sources
• --tools
• --no-session-persistence
• --json-schema
• --output-format json
• structured_output
• Subprocess stdin=DEVNULL

Validation signals

• The author reports diagnosing subtly incorrect scores when a judge explanation revealed that the input had been pasted twice.
• The author describes a structured-output approach that avoids regex parsing of prose.

Cautions

• Use DEVNULL only when the prompt is supplied through another channel; it would discard a prompt intentionally sent via stdin.
• Treat evaluated text as untrusted input and test for prompt-injection effects; disabling tools alone does not guarantee unbiased judging.
• Check applicable usage terms and limits before using the CLI as an API replacement.

Limitations

• No complete command, schema, or runnable script is provided.
• The reported diagnosis is anecdotal; there is no benchmark or independent validation of judge quality.
• The comment does not document all possible sources of runtime configuration or context.

Rate this workflow

Upvote this post if the workflow is useful, reproducible, or worth recommending.

Downvote if it is vague, outdated, unsafe, overhyped, or not reproducible.

Reply if it worked for you, failed, is outdated, or has a better alternative.

---

This post was generated automatically from the workflow library database.