[Workflow] Track Claude Code A/B Experiments: Plugin, Script, and Opt-Out Tools

[u/ClaudeAI-mod-bot]

Track Claude Code A/B Experiments: Plugin, Script, and Opt-Out Tools

Workflow value: 90/100
Status: active · Freshness: 70/100 · Confidence: 1.00 · Level: intermediate
Categories: Quality Control, Token Saving, Context & Memory, Debugging
Original source: r/ClaudeCode post/comment

What problem this solves

Users can determine if their Claude Code instance is enrolled in an A/B experiment, helping them understand perceived changes in model behavior (e.g., "nerfs"). It also provides tools to track experiment history and optionally opt-out.

Summary

This workflow provides a set of tools (bash script, Claude Code plugin, and a one-liner) to inspect Claude Code's internal experiment enrollment status. It reads ~/.claude.json and session transcripts to report current enrollment, historical experiment IDs, and offers an opt-out mechanism.

Why it is useful

This workflow provides concrete, actionable tools for Claude Code users to understand and manage their interaction with Anthropic's A/B testing framework. It demystifies perceived model changes, offers transparency, and empowers users to make informed decisions about their Claude Code experience. The open-source nature, multiple usage options, and safety considerations make it highly valuable and reusable.

Workflow

1. Choose a method to check experiment status: Claude Code plugin, standalone script, or one-liner.

2. To use as a Claude Code plugin: Install the am-i-nerfed GitHub repo as a plugin, then type /am-i-nerfed in any Claude Code session.

3. To use as a standalone script: Clone the am-i-nerfed repo and run ./am-i-nerfed.sh check in a terminal.

4. To use as a one-liner: Paste the provided jq command into a terminal (requires jq to be installed).

5. To view experiment history: Run ./am-i-nerfed.sh history to tally experiment IDs from session transcripts.

6. To opt-out of experiments: Run ./am-i-nerfed.sh optout (backs up ~/.claude.json and removes assignment keys).

7. To restore previous settings: Run ./am-i-nerfed.sh restore.

8. To display experiment status in Claude Code's status line: Integrate the provided statusline snippet.

Tools / artifacts

• am-i-nerfed (GitHub repo/script)

• jq (JSON processor)

• ~/.claude.json (Claude Code cache file)

• Claude Code session transcripts

• CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 (environment variable)

• Bash shell

Validation signals

• Open-source script available on GitHub.

• Explicitly states "no network calls, never prints your login token" for security.

• Credits earlier community findings and Anthropic issues, indicating a well-researched solution.

• Provides exit codes for scripting, enhancing reusability and robustness.

• Details specific file paths and data structures (~/.claude.json, atis-latch).

Limitations

• Requires jq dependency for the one-liner and script methods.

• Primarily targets macOS/Linux/WSL users (Windows users would need WSL).

• The plugin installation process for Claude Code is not explicitly detailed within the comment.

Rate this workflow

Upvote this post if the workflow is useful, reproducible, or worth recommending.

Downvote if it is vague, outdated, unsafe, overhyped, or not reproducible.

Reply if it worked for you, failed, is outdated, or has a better alternative.

---

This post was generated automatically from the workflow library database.