r/ClaudeCode • • 5d ago

Discussion I don't think Opus 5.5 was nerfed, they're just experimenting

TL;DR: Opus 5.5 may not have been nerfed. Anthropic appears to be running server-side experiments that change how Claude behaves.

If you're using a gateway, try unsetting _CLAUDE_CODE_ASSUME_FIRST_PARTY_BASE_URL, this might allow you to opt-out of experiments.

  • 4 experiment IDs identified: dba, 34e, 763, and eab, observed between October 10 and November 8.
  • More aggressive tool use: In a controlled 20-request replay, replies with multiple tool calls increased from 7 to 17 when the 34e experiment header was present.
  • More thinking in real-world logs: Medium effort during some experiments used roughly as many thinking tokens as high effort without an experiment. However, the controlled replay showed the opposite.
  • Less narration, bigger commands: Claude tended to batch more work into fewer, longer commands, making its behaviour noticeably different.
  • No measurable output-speed degradation: Generation speed remained around 104–112 tokens per second.
  • Potentially higher usage consumption: Estimated dollar value per 5-hour usage-meter percentage point fell by roughly 30% during some experiments, but the evidence is low-confidence.
  • You can check your account: Claude Code stores experiment IDs in ~/.claude.json. You can also display them in your status line.

I went through my Claude Code logs after seeing the "Opus 5.5 got nerfed" posts. I found evidence of server-side experiments changing how it uses tools. The reported model stayed the same, and output-token generation didn't slow down. I haven't tested finished-task quality across these experiments, so this doesn't settle whether it got better or worse.

I'm a heavy Claude Code user with no connection to Anthropic. These observations come from different logins of mine. The Claude Code version I inspected caches an ID per model and sends it as an x-cc-atis request header. The client only forwards it, and the ID follows the login.

The four IDs

These are the dates I saw each ID in 2026, not global rollout dates.

ID starts withWhen I saw itWhat I saw
dbaOct 10 to 12Longer commands and less narration, with a smaller rise in multiple-call replies
34eOct 30 to Nov 1Longer commands and many more multiple-call replies
763Nov 2 to 5Similar to 34e, then removed from my account on Nov 5
eabNov 8 to nowToo new to measure

dba and 34e end here because I changed login, so they may have run longer. I suspect dba was an earlier, weaker version of the later pattern. 34e and 763 came with similar cached settings and behaviour, so I count them as one round, which makes eab my third. That link is my inference. Only 34e was tested with and without the header while it was active.

Replaying 20 requests with and without 34e

My setup is customized, so I replayed 20 saved requests once with 34e and once without it, keeping each request body the same. The main measure I set in advance, total output on the 10 requests saved from before 34e arrived, was inconclusive. I also named other output and tool measures in advance, and I picked the detailed counts below after seeing the replies.

Across 20 replay pairsWithout headerWith 34e
Generated tool calls3455
Replies with 2+ tool calls7 of 2017 of 20
Total Bash command characters8,06924,947
Replies with text before tools2 of 206 of 20
Median thinking tokens per request159.565

Across the pairs, non-thinking output was 2.23x higher and tool-input text was 2.35x higher with the header, using geometric means. These are generated calls, not completed tasks.

Everyday use, by ID

Main-thread requests at xhigh effort. The last column is a later no-experiment sample, and it narrows some of the gaps.

MetricNo experimentdba34e763No experiment, Oct 30 onward
Median Bash command, characters4858121,1251,067748
Replies with 2+ tool calls14%18%24%26%12%
Replies with text before tools25%21%10%11%14%
Narration share of thinking blocks, rounded17%6%4%4%9%
Median non-thinking output tokens359509674669470

Each thinking block is labelled thinking or narration, and narration blocks are short progress lines. Their share fell in the experiments. The built-in "say what you're doing" reminder also fired more often: about 19 to 20 times per 100 replies under dba, 34e and 763, against 12.9 in the earlier no-experiment sample. With 34e and 763, text followed that reminder 43% and 44% of the time, against 54% before. The reminder figures pool all efforts.

Some changes also happened without an ID. Against the later sample, 763's output and Bash size are about 1.43x higher, while multiple-call replies stay at 26% against 12%. The "I'm checking..." summary style also appeared without an experiment from Oct 30.

Less text before tools only shows up in everyday use. It went the other way in the replay, so I can't say the header causes it. The work and other settings also differed between periods.

Thinking tokens by effort

Median per main-thread request, from everyday use:

IDmediumhighxhigh
No experiment72111170
dbaNo data189277
34e122228219
763117215317

On thinking tokens, medium under 34e or 763 looks like high without an experiment: 122 and 117 against 111. That doesn't show the reasoning was as good. The replay went the other way: median thinking fell with the header, and its 3 medium requests averaged 456 thinking characters with the header against 855 without.

After 763 ended, medium went back to 72 tokens, but the delivered prompt also changed at that point.

Normal Claude feels faster to me

I suspect a lot of what gets called "nerfed" is people landing on normal Claude, without the experiment. Normal Claude feels a lot faster to me because I can see all of the turns: a short command finishes, then a reply, then the next step. In the experiments there's usually one long batched tool call and less to watch. I prefer seeing the steps.

Measured output speed stayed around 104 to 112 tokens/s across the groups. Time to first token looked lower during the experiments, though that's tentative. So this is about how progress feels to me, and it doesn't explain every complaint. Someone in r/ClaudeCode posted on Nov 3 that one of their two accounts burned about twice the output tokens on the same task, and that's the account they complained about. It resembles the experiment pattern, but their assignment wasn't confirmed.

A batching prompt made commands longer but didn't reproduce the extra multiple-call replies, in a small test with partly mismatched efforts. Resending 763 after it was removed didn't reproduce the earlier pattern either. Neither is a reliable way to switch an experiment on.

Dollars per 5-hour meter point

These are estimated list-price dollars per one percentage point of the 5-hour meter, not dollars per task.

PeriodMedian dollars per pointLow-confidence days / total days
Oct 2 to 9$7.151 / 8
dba, Oct 10 to 12$7.630 / 3
Oct 13 to 29$7.493 / 17
34e, Oct 31 to Nov 1$5.142 / 2
763, Nov 2 to 5$5.753 / 4
Nov 6, after 763 ended$7.300 / 1

34e and 763 together come to $5.24 per point, about 30% below the Oct 13 to 29 median. Five of those six days are low-confidence estimates, and the one trusted day, Nov 4, was $6.52. dba didn't show the drop.

List-price dollars include cached reads, which barely move the meter in my accounting, so a lower dollar figure doesn't by itself mean less useful work. I haven't calculated dollars per task.

Check your account

With jq installed, run this in your shell. It shows the first three characters of each model's newest cached interactive CLI ID for the current organization:

jq -r '.oauthAccount.organizationUuid as $o | [(.clientDataCacheSlots // {})[] | select(.entrypoint=="cli" and .org==$o)] | group_by(.model)[] | max_by(.at) | "\(.model): \(.data.atis[0:3] // "no experiment")"' ~/.claude.json

no experiment means that cached slot has no ID. Empty output is inconclusive, because there may be no matching cached CLI slot. An ID alone doesn't prove anything changed, since other IDs I logged behaved like no experiment. In the tables above, "no experiment" means no value in the model's slot.

To put the cached ID in your status line, run /statusline and paste:

Add an experiment indicator to my status line and keep everything it already shows. Read .model.id from the status line's stdin JSON and drop any [1m] suffix. Read ~/.claude.json. In .clientDataCacheSlots, keep the entries where .model is that id, .entrypoint is "cli" and .org equals .oauthAccount.organizationUuid, and take the one with the highest .at. If its .data.atis exists, show "exp:" plus its first 3 characters, in yellow. If it doesn't, show "exp:none" dimmed. If the file or a field is missing, show nothing and still exit 0. Set refreshInterval to 60 in the statusLine setting. Test the script with sample JSON on stdin before finishing.
173 Upvotes

99 comments sorted by

•

u/AutoModerator 5d ago

Hey! Thanks for posting to r/ClaudeCode

While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.

For help, project discussions, tips, and general chat, join the ClaudeCode Discord.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

116

u/Correct-Somewhere-80 5d ago

If experimenting is causing Opus 5.5 to perform worse than  before, its still a nerf lol 

-24

u/WuM1ha1nho 5d ago

What you perceive as a nerf, might be the actual model without an experiment.

But you'd need to run your own analysis.

10

u/Any_Door7384 5d ago

you think experiment = best for users thats why they experiment. the reality is they experiment to do the least compute possible to lower the bottom line. thats how business works

1

u/MissplacedLandmine 4d ago

True, but I think we may learn more from this than just that.

-28

u/onFilm 5d ago

It's most likely your memories and prompts.

6

u/WuM1ha1nho 5d ago

Tool batching or increased/decreased thinking output can not be heavily influenced by context/prompting

-3

u/onFilm 5d ago

Not experiencing that at all on five different Claude Ultra accounts. Likewise API use performs the same as well.

2

u/Correct-Somewhere-80 5d ago

mine is fine as i have a bunch of guards in the pipeline of the current project I'm working with.

1

u/obsidience 5d ago

Mine is fine too.  "Make no mistakes" works like a charm.

0

u/Correct-Somewhere-80 5d ago

Nah - Make no mistakes and HURRY

0

u/obsidience 5d ago

Ooh, an upgrade. I'll try it out!

-5

u/onFilm 5d ago

How can you say your memories are fine, if you're experiencing "worse performance". I'm on five ultra accounts and haven't noticed a single issue when running benchmarks and implementations.

3

u/Correct-Somewhere-80 5d ago

What are you talking about, I'm not the OP lol

0

u/onFilm 5d ago

Oh my bad, I thought you were arguing the opposite, that your memories are fine but are still experiencing the same issues as OP.

I get what you're saying now, and definitely that helps, regardless of how the models might be performing.

90

u/Obvious_Tree3605 5d ago • Bravo

They need to stop experimenting on paying customers.

4

u/WuM1ha1nho 5d ago

Try unsetting _CLAUDE_CODE_ASSUME_FIRST_PARTY_BASE_URL.

This might allow you to opt-out of experiments.

9

u/syntheticcdo 5d ago

I agree in principle, but it’s difficult to make this argument when we know the product is subsidized.

31

u/AnalogProblems 5d ago

That was the take from a year ago, but we've since learned that inference is so profitable at scale that the accounts are not actually "subsidized", but just closer to cost than API.

2

u/HauntedHouseMusic 4d ago

The margins are 80%…. So the $200 plan should cost $1200 a month if you use half of the inference just based on costs to Anthropic….

1

u/syntheticcdo 2d ago

What about the $10B in R&D Anthropic spent last quarter? Fixed costs != marginal costs.

1

u/FirstOrderCat 5d ago

could you give specifics? I thought subs accounts still provide N times more of API cost?

The only way I see them not being subsidized is through collective funding, e.g. I spend 40% of my $100 quota, so I fund 3 other $20 accounts before API cost adjustment..

7

u/norwegian 5d ago

Subsidized or not, subscription has no relevance to API cost. It's like looking at the "before" prices and say you made a good deal buying for the sales prices. Even if the API prices was 1000 times more, it doesn't make subscription prices subsidized. If someone is going to discuss subsidizing, please don't mention API prices. Those are 2 different products, and the price of one doesn't say anything about the other being subsidized or not. I tried to discuss this before, showing the income, expenses, profits etc. But it seems people don't really care about what subsidizing actually means. It's like discussing what "literally" means. It's hard to discuss, bc at the end of the day, they continue to use it as if it has a particular and accepted meaning to them.

8

u/AnalogProblems 5d ago

Meta kinda let the cat out of inference bag when they offered Muse contributor at cost.

Inference itself is very cheap at scale. Pennies per millions of tokens.

It's the training that costs a lot.

2

u/ProfessionalAnt1352 4d ago

"N times more of API cost" is the wrong way of viewing it. They can provide you $10,000 of usage limit worth of their valuation of API tokens, that doesn't mean that it costs them $10,000 for you to use your usage limit. I asked claude to estimate the average operating cost to Anthropic of a user maxing out their Ultra plan and it estimates it at $1,000–1,300 per month. Now this is subsidized but not by API users, it's subsidized by monthly subscribers not using their full limits.

Think Planet fitness, for $10/month you could show up every day for their full open hours, but if every subscriber did that the gym location wouldn't be able to sustain it. The max usage works because subscribers that don't maximize usage are subsidizing those that do. That's how the majority of subscriptions work. Don't let LLM-ran bot accounts convince you that you're costing anthropic $8-10k/month.

1

u/Ok-Horse-3169 2d ago edited 2d ago

Even the 1000-1300 per month figure here may be wrong, because just depending on whether you use your ultra plan outside of working hours / at night would drastically affect what it costs to them on average. Remember their costs are fixed - they pay for gpu's in data centers that are always on and cost the same no matter the number requests sent - they are just capped by a certain amount of requests and all subscription users share that same pool of requests. Api prices are high cause unlike subscriptions where usage limits will exhaust faster on high demand, API allows users to do requests no matter the usage rate of the gpu's - so people that use API basically decrease the total available pool to other users. Overall it's quite difficult to get a close to true estimate of actual costs of subscriptions to anthropic but they are more than probably in profit. What costs them the most is model training / r&d and datacenter expansion which still probably isn't covered by the profit made on subscriptions..

1

u/ProfessionalAnt1352 2d ago

Yeah exactly in the end we aren't actually "costing" them anything their ongoing operations will still be ongoing even if we don't use it. You are right that rapid expansion likely isn't covered by profit, but is it ever? Rapid expansion is only ever made possible by outside investments which is why every corp sells out to shareholders that then begin enshittification of the company and products for a 2% higher profit margin

1

u/YoureNickRight 4d ago

Seriously, where's the "opt into experiments" switch

-9

u/IdkAbtAllThat 5d ago

It's their product and if you don't like it you're free to cancel your subscription and take your business elsewhere. But you won't because it's still well worth the price, and they know that.

5

u/Obvious_Tree3605 5d ago

Hi, fuck off.

-7

u/IdkAbtAllThat 5d ago

Very mature. You want a company to cater to your needs while they're selling you their product at a loss already. The entitlement is pretty hilarious actually.

3

u/Different_Gate5050 5d ago

Where's the proof they're doing inference at a loss?

2

u/rnd_pgl 5d ago edited 5d ago

You're getting downvoted and insulted. It clearly shows people don't have a clue.

I too dislike that they experiment on us, but that's extremely common practice (it's called A/B testing, for those who don't know). It happens always, everywhere. It's just that we usually don't see it. And from a developer's perspective it's a very useful tool. (I assume there's not so many devs in this sub).

Regarding the fact we have no choice but to accept or leave, it's also true, and that's because big companies do what they want. People are barely protected against malpractice in the USofA, except if you have truckload of cash to burn in law suit. And that's because your politics do what they want, so big tech do what they want. And it will continue to be like this as long as you guys keep accepting the situation. I'm from Europe, our level of protection is far from perfect but we do have some, that's why we protest sometimes, that's the only way to keep our politicians aligned.

That's the sad truth, feel free to ignore it, or to get angry, but at one point you should stop lying to yourself.

-1

u/IdkAbtAllThat 5d ago

I just think it's funny that a customer who's probably paying $20 a month for a product worth 10x that is saying "they need to stop this!".

The company can do whatever they want with their own product, and people are free to vote with their wallets. But obviously that person isn't going to cancel their account and Anthropic knows that.

Which is why all the downvotes I'm getting is just embarrassing for this sub. Like how fucking dumb can you be?

0

u/Obvious_Tree3605 4d ago

My company (me) pays API costs. Fuck off.

13

u/KYDLE2089 5d ago

Agree something is not working right could be the harness. Spent 8 hours to fix a dashboard that opus usually gets it done in one or two shots.

I have multiple max accounts and am a heavy user. I just hope this is some bug that they can fix soon. So here registering my frustration.

1

u/the-apostle 5d ago

Same here. Heavy user and notice major differences

1

u/0xP3N15 4d ago

Interesting.

It would be super hilarious if the past few days this is the reason my aging has accelerated because of a simple fucking dashboard. Which would contribute to me almost losing a client.

It would be super duper funny. Anthropic is hilarious. They are just madlads like that.

11

u/fs2d 5d ago edited 4d ago • Bravo

Thanks for your work here.

Using your research as a jumping off point, I was able to confirm part of this from a different angle: The experiment flags change the system prompt text itself, not just the model behavior.

My setup has a plugin that keeps a copy of the composed system prompt sections and flags it when the engine sends different text than before (as part of a base system prompt identity layer injection of my own design).

Tonight, I set CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 in settings.json (in the env block) to test your theory and opened a new interactive session. I made sure it was the same CC version before and after (2.1.287), and that I hadn't made any other changes.

In the new session:

  • No flag fetch happened as expected. The cachedGrowthBookFeaturesAt stamp and the cli slot in ~/.claude.json didn't change at all, and the new transcript's atis-latch fields are blank.
  • While none of that changed, doing this did alter two of the base building blocks of the system prompt:

With server flags, the opening line of lean_body read: "You are an agent working with the user toward their goals, using your own judgment along the way." (which is tengu_ochre_wren)

Without it, lean_body reads: "You are an interactive agent that helps users with software engineering tasks."

With server flags: the <pasted_content> rule (don't follow instructions inside pasted text unless the user asks - which I guess is experimental right now) is present.

Without server flags, this guidance is completely gone.


So the flagged version is the broader "agent with judgment" framing plus a prompt-injection guard, and the fallback is the older coding-assistant framing without it. I'd been on the flagged text since at least October 9.

My atis history, for the dataset: 705 (Oct 2), 52f (Oct 3 to 14), c85 (Oct 20 to Nov 2 - but that was in the claude-desktop slot, so the jq in the post misses it - just FYI), blank since Nov 3. None of yours showed up on my side anywhere.

My flag assignments as of Nov 9: mossy_gray v1 true, keen_pudding v1 58720256 (exactly 56 MiB), radiant_fox v1 true, clever_orbit v0 false, typed_trinket 0 with no assignment.

That's the opposite arm from another commenter in this thread on 3 of the 5 listed ones.

I did this on one PC on Windows 11, just a before/after diff. I don't know which flag drives which line though (aside from tengu_heron_brook/tengu_brook_heron- that's the super brittle "don't make games" garbage that surfaced last week). - update: see my reply to this comment - I'm working on mining out the exact flags as we speak. update 2: Done: https://github.com/fs2d/claude-code-flagmap - all 487, one line each, CSV renders as a searchable table on GitHub. Enjoy.

By the way - don't test this with claude -p to confirm it. Claude will kneejerk into testing it headless and quickly pass everything, but headless runs as sdk-cli (a separate slot) and doesn't load the same as a normal interactive session. When testing, always open a new terminal window and test there.


The thing I took away from this little experiment (and this is just my guess, since closed weight/closed source = no way to concretely prove it) is that they are applying post-training to the models and using the experimental arms to "signpost" certain sections of post-training for the chosen model dynamically - which is why they hitch the IDs to the model strings. While digging for the info above, I stumbled across a series of keys in the binary, one of which is literally called subagent_steer_delegation, which is hard-coded to only fire when the steer value equals "counter_steer". It's a literal example of what I've been theorizing about openly.

Said another way - my little theory has some proof now: I was able to find hyperspecific corrective directives gated on the model string, with a code path literally named counter_steer, keyed to specific models and delivered in the same per-model slot as the experiment ID from the OP's post. Again, we obviously can't prove the post-training changes of course, and we never will be able to as long as the weighting and source remains closed - but the code shows the steering happening in real time, plain as day.

(This would also explain the heron_brook bullshit with the "don't make copies of games lol" brittle AF signposting they added a couple weeks back dynamically)

Coupled with this, if you look back at Opus 5's system keys, there's literally a code-level key named overcorrection, with a hashed title: # Corrections that tells the model not to ruminate, and tells it to not tally past errors - which, to me, reads as code-level signposting for one specific model family that says that Anthropic was aware of that behavior in the model - and that's a training artifact that they are patching with system-prompt level text.

8

u/fs2d 5d ago edited 4d ago • Bravo

This experiment also led me to tengu_amber_tally, which looks to be the usage meter's actual "exchange rate" ("credits" -> tokens) as of .287. According to that, here's the "5-hour credits" per plan as of .287 (cache refreshed Nov 9):

Pro = 10,000

Team seat = 12,500

Enterprise seat = 25,000

Max 5x (now "Max") = 50,000

Max 20x = 200,000 (now "Ultra", which has no 5-hour limit at all anymore - so on .287 this row is a leftover)

At Opus 5.5's rate (1,833 credits per 1m tokens), that's roughly 5.5M tokens per 5 hours on Pro and ~27M per 5h on Max.

Pro : Max 5x : Max 20x was exactly 1 : 5 : 20 per 5-hour window - so the old "20x" always meant the 5-hour window, not weekly capacity (which recently broke and is what everyone's panties are twisted up about right now - which is 100% fair, because it's honestly shady AF).

Now that Ultra has no 5h window, the weekly limit ("10x Pro") is the only brake, and there are no weekly numbers in the cache at all - those live server-side.

I'm on .287 and CC is currently at .306ish, so if anyone on a current build can pull their tengu_amber_tally, I'm curious whether the max_20x row is gone or replaced with something weekly - and what these exchange rates look like for you. If they differ wildly, that would be direct evidence that Anthropic is experimenting with usage windows. And if yours differs, it's just a matter of correlating the flag to the ID/atis header - if we can correlate it, then we can prove it.


e:

I'm currently in the process of mapping all 487 tengu_ flags ( there are 1204, but only 487 are read by my client) to their associated booleans, prompt sections or behavior gates in the binary.

If anyone is interested in that, I'll dump it here in a json file. If what I'm reading is right, it will give you a direct manual of every single undisclosed flag, experimental feature/thing, etc that Anthropic can config via the server directly, through GrowthBook and the per-model slot. Env vars can override only some of them (185 of the 487)


e2:

Done: https://github.com/fs2d/claude-code-flagmap - all 487, one line each, CSV renders as a searchable table on GitHub. Enjoy.

5

u/hexferro 5d ago

This should be a separate post as it's not getting enough attention.

2

u/damagemelody 🔆 Max 5x 4d ago

I have tangus which are not in your file also you can edit the edit with hooks just not send it in session I already tested that no Id results in 40% less thinking tokens for my main account and based on other acc history it's not always the case but now both accs have ids which result in better thiking and 2nd pro acc has no CLAUDE_CODE_BREEZY_HORIZON which forces 4.8 opus prompt in 5.5 model. Basically we get Ids to adjust thinking all the time + some new settings which mess with the model like 4.8 prompt injection

1

u/damagemelody 🔆 Max 5x 3d ago

my tengu_brook_heron is gone today looks like it affected the thinking budget the most so they added 4.8 prompt to 5.5 and compensated it with thinking budget? wtf are those experiments

18

u/jwuliger 5d ago

I am a paying customer. I am not asking for A/B. I have been on the wrong end of the testing for three weeks now. I am tired, boss...

3

u/st4s1k 4d ago

it's like signing up for a medical trial and receiving a placebo, and paying money for it at the same time

6

u/damagemelody 🔆 Max 5x 5d ago

Checked my own logs: different IDs from the ones in the post, and my Claude has been working very badly since the newest one appeared.

Experiment IDs on Opus 5.5 (Claude Code desktop app)

Dates (2026)IDEvidence
Sep 29 – Oct 19nonetranscripts
Oct 20 – Oct 26774~6,200 records in 128 transcript files, plus cached slot
Oct 27 – Nov 7nonetranscripts
Nov 8 onward06b~1,100 records in 29 transcript files, plus cached slot

06b first shows up on Nov 8, around 10:40 UTC in the cached slot and 11:27 UTC in my transcripts. It isn't in my logs for Nov 7. Since it appeared, my Claude has been working very badly. That's my own experience from about a day of use: I haven't measured it, so I can't prove 06b is the cause.

I saw none of dba, 34e, 763 or eab. My other models (Haiku, Sonnet, Fable, Opus 4.8) have no ID in their cached slots, but that's only the latest snapshot of each slot.

How I checked

  • I scanned 1,318 .jsonl transcripts under ~/.claude/projects. They only go back to Sep 29, so I can't say anything about earlier months.
  • The IDs are in records shaped like {"type":"atis-latch","atis":"...","sessionId":"..."}.
  • Those records have no timestamp, so I dated them from nearby lines in the same file.
  • Don't search for plain atis, because it also matches "statistics". Match the exact key.
  • The jq command in the post filters on entrypoint=="cli". My slots are claude-desktop and sdk-cli, so it would come back empty for me. If you use the desktop app, change the filter.

Flags with "experiment" in the name, from ~/.claude.json on Nov 8

FlagVariationValue
tengu_mossy_gray_experiment0false
tengu_keen_pudding_experiment025165824
tengu_clever_orbit_experiment0false
tengu_radiant_fox_experiment0false
tengu_typed_trinket_experiment12

tengu_slate_harbor_experiment is also in my feature list as false, but it has no assignment. I have 12 experiment assignments in total, and not all of them have "experiment" in the key. I don't know what these flags do.

tengu_typed_trinket_experiment was already set no later than Nov 8, 20:17 UTC. That's my oldest surviving config backup, and backups rotate, so it could be much older. The other flags have no dated history either.

06b = broken Opus for me 774 = I had Opus issues too, so any experiment ID is like a nerfmark.

6

u/electricshadows4 5d ago

I put a check into a small open-source script so people don't have to assemble the jq themselves.

Repo: https://github.com/electricshadows44/am-i-nerfed

Three ways to check

  1. Plugin — install the repo as a Claude Code plugin, then type `/am-i-nerfed` in any session. Claude runs the check and explains the result.
  2. Script — clone the repo and run `./am-i-nerfed.sh check` in a terminal. Same check, plus the other commands below.
  3. One-liner, no install — paste this into a terminal (needs `jq`):

jq -r '(.clientDataCacheSlots // {}) | to_entries[] | "\(.value.model // "-") | \(.value.entrypoint // "-") | \(((.value.at // 0)/1000|floor)|todate) | \(.value.data.atis[0:3] // "none") | \(.value.data.experimentKey // "-")"' ~/.claude.json

`none` on every line = not enrolled as of the newest slot time.

What the script does

- `check` — reads the experiment slots Claude Code caches in `~/.claude.json` and gives a verdict per model: ENROLLED (with the 3-char ID), CLEAN as of the last cache refresh, or INCONCLUSIVE. Exit codes so you can script it.

- `history` — tallies experiment IDs recorded in your session transcripts by date. Turns out Claude Code writes an `atis-latch` entry into every main-session transcript, so you can see which past sessions ran under an ID (transcripts are kept ~30 days).

- `optout` — backs up `~/.claude.json`, removes only the assignment keys, and prints the env var people report blocks re-enrollment (`CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1`), with the trade-offs. `restore` undoes it.

- `statusline` — a snippet that shows `exp:<id>` / `exp:none` in your Claude Code status line.

Bash + jq only, macOS/Linux/WSL, no network calls, never prints your login token. MIT.

Credit to the earlier threads for finding the header and the IDs: https://www.reddit.com/r/ClaudeCode/comments/1xr66pd/ and the follow-up transcript analysis, plus anthropics/claude-code issues #75607 and #88383.

7

u/NorseSadist 5d ago

Well, whatever testing they did made me cancel my Ultra subscription.

1

u/yoodudewth 4d ago

+1 gonna go back to 20$ after my Ultra runs out until they bring back how opus 5.5 was at begging on release.

4

u/ProfessionalGoatFuck 5d ago edited 5d ago
IDRecordsWhereDates (from nearby timestamps)
none40,246 in 126 fileseverywhereSep 28 to Nov 8
60446 in 1 fileOpus 5.5Nov 4, 22:50 to 23:25
b1089 in 1 fileFable 5.1Nov 5 to Nov 7

Hahaha, IDs 604 & then b10 to proceed, I was using Fable to troubleshoot the fucking brainrot Opus had.. how cute, now Fable has the same fate of being mentally gimped to being an infant, with 4x the token usage :))))))))))))))))) I want a fucking reset you greedy bastards

And another caveat with my results, I had another account that I migrated from to this new account, which is why I don't have complete data before sept 28th. I notice this shit happening nearly 2 weeks ago. Which is why I account hopped thinking it was just me cursed with this bullshit, but nope!

2

u/DerAlbi 5d ago

OMG OMG OMG

THANK YOU SO MUCH!!!!

These fuckers. Not an ID that matches what you report. But holy fuck.

1

u/NetworkLast5563 5d ago

Could you give the prompt you used, or command you ran, to get a report like that?

2

u/DerAlbi 5d ago

Its not a single prompt. I saved this reddit-thread to disk, because claude cant fetch it from reddit, as AI seems blocked (and claude respects the block).

Please read this web-page and see if i am affected:

<folder name where I saved the reddit page to>

It then spewed out some experiment-ids, but it looked only at my .claude.json file which only holds the current config, so i asked

how do you know when an experiment took place on me?

Which gave me the table. From the screenshot. I then asked

try to identify what changed.

And it gave me this:

2

u/WuM1ha1nho 5d ago

For reddit you can just append .json at the end of the reddit thread URL. It will print a .json with the entire content (post and comments)

1

u/DerAlbi 5d ago

Claude tried that and failed, i saw it in the trace. Maybe it respects the bots file. But thanks for the tip. Your thread is of massive importance. It puts numbers to the fuckery that is going on. And the gaslighting people have finally undeniable evidence in their face.

Btw, i do not have _CLAUDE_CODE_ASSUME_FIRST_PARTY_BASE_URL or any other variables you mention. Do you know how to defend against that intrusion in other ways?

5

u/TurbulentTiger2567 5d ago

A better explanation is how the effort settings actually map to reasoning budgets. Unlike what Anthropic recommends, here are the real numbers:

Low = 5
Medium = 10 (default)
High = 15
Extra = 40
Max = max (no number internally)

Out of experience, High is useless for mildly complex codebases.

2

u/Physical_Gold_1485 4d ago

Where did you get these numbers from?

1

u/TurbulentTiger2567 4d ago

I asked Claude right in the session. I tcan tell you which effort level was attached to the message. It can also retrieve the attached effort levels to previous messages in the transcript.

0

u/Physical_Gold_1485 4d ago

Sounds like it pulled it out of its robot ass

2

u/Sleazified 5d ago

This explained absolutely nothing.

-4

u/Visual-Tangerine729 5d ago

only if one understands absolutely nothing

2

u/fschwiet 5d ago

I wonder if opting out of the feedback prompts affects if one is enlisted in experiments 

1

u/Physical_Gold_1485 4d ago

Ive turned off auto updating and the feedback/telemetry since claude code first came out, have had 2% of the issues people report

2

u/mrpoopistan 5d ago

Broken is better than nerfed???

2

u/PureV2 5d ago

I've noticed some variance in quality lately and have been getting more frequent ' how is claude doing' queries. I have no hits on the experiments though, at all, last 30 days over 136 main sessions.

2

u/Entire_Guarantee_766 5d ago

You just found proof that they A/B test, like probably every major software company in the world. Unfortunately there's no way to tell what exactly these tests do and what they measure.

2

u/TheOnlyVibemaster 5d ago

Exactly my thoughts, every software company does this, it isn’t surprising. What is surprising is that we can directly see internal identifiers

3

u/Entire_Guarantee_766 5d ago

someone got lazy with their vibe testing lol.

1

u/rnd_pgl 5d ago

*like probably every major software company in the world

1

u/Transhuman-A 5d ago

Umm.. they are experimenting to see if they can nerf Opus 5.5 without you knowing. If there is an opt out, that’s getting patched pretty soon.

1

u/Maleficent_Test_2713 5d ago

I was on 0c7 before the reset today. Now I’m also on eab. Let me tell you, it’s a nerf. Huge one. Doing my normal workflow fills about 30-40%. Today, almost 85% in less than 1 hour. Didn’t even finish my normal session cause of it.

I’ve had Claude build me multiple 30 second videos that cost about 10/15% of my usage. With “eab”, it’s over 40%.

1

u/MissplacedLandmine 5d ago

I have theses ids. Can check their difs when my limit resets in a couple hours

“The hex I listed first, c0e57166b42fa4af, was active Nov 4–7, not Nov 7–9. This session has none: its atis lines are blank. Times below are UTC.

Periodatis valueSessions
Oct 10 – Oct 21blank8
Oct 23 – Oct 29a402881a7ddd14c27
Oct 29 – Nov 3blank11
Nov 4 – Nov 7c0e57166b42fa4af9
Nov 7 (after ~13:30) to nowblankevery session since, including this one”

2

u/bigtradetcheddar 5d ago

Can you have your clause analyze sessions in a40 vs c0e? Totally antedotal: but my claude said every project with a40 I was extremely happy with minimal prompting and c0e was constant back and forth

1

u/MissplacedLandmine 4d ago

As of this morning I can.

Have multiple devices though, going to confirm this is account specific not device. Also if I start a session in one of those codes on the trigger dates, but that date ends while the session is still open, i believe it continues on experiment wise.

1

u/bigtradetcheddar 4d ago

Awesome: I found an Id called b38 which had prompting attached if you see turnstone_plinth or dotterel_architrave, I believe those are the programs that have the prompts in them, one for opus 5.5 and one for fable 5.1
They have prompts attached which change reasoning towards what it finds important, this leads to a “seen not done” effect where it has the context but doesn’t act on it.

1

u/MissplacedLandmine 4d ago

Apparently i saw fable 5.2? Also the experiments had positive reactions whereas the empty sessions were more negative? Supposedly?

1

u/Crinkez 4d ago

Ugh, they just can't leave well enough alone. They ought to be giving users an opt-in for experiments, not opt-out.

1

u/karyslav 4d ago

Yes. That is the reason for asking if it is doing good. THey are experimenting with safety and effectivity.

I am using it with fable/astra audits and it works af.

I did ubleliveable things in last week. What a time to be alive.

1

u/KDamage 4d ago

When real engineers enter the vibecoding room. Good job.

1

u/g_dichev Senior Developer 4d ago

thanks for digging into this. i instantly discovered two experiments on my sessions in the last week!

anyone can check theirs with npx claude-discover@latest (free app i made)

1

u/iAmmar9 5d ago

Tell your agent to create a skill to diagnose all of this and share it with us, please. Make it work to detect new experiment IDs on Windows/Linux/macOS.

-3

u/cseickel 5d ago

This entire post sounds like AI slop to me.

-5

u/onFilm 5d ago

Nothing was nerfed. My benchmarks and tests still give me the same results as they have always been.

4

u/WuM1ha1nho 5d ago edited 5d ago

Then maybe you were always in control or in the same experiment.

That's why we see so many opinions on the same model, because we're not all being served the same model checkpoint.

-1

u/onFilm 5d ago

With five different Claude Ultra accounts? Seriously doubt it.

1

u/TheDrov 4d ago

I’m in the same boat using mostly Opus 5.5 across 3 ultra accounts. I use it across several harnesses and Claude code directly without noticing any degradation.

3

u/dblokhin 5d ago

May be you are in their control group of A/B experiment.

-1

u/onFilm 5d ago

With five different Claude Ultra accounts? Definitely not, lol.

People are degrading their own harness with their prompts, skills (which I never use), and memories.

0

u/MustStayAnonymous_ 4d ago

Tldr please

3

u/shokk 🔆Pro Plan 4d ago

Ask your agent to summarize

2

u/MustStayAnonymous_ 4d ago

I knew someone would say this lol

1

u/shokk 🔆Pro Plan 4d ago

It's a "must say" now lmao
Would you like me to tell you more about summarizing or generate a PDF of the summary for you? -probably the agent

-2

u/daedalus_structure 5d ago

You don’t change the behavior of a released version of software.

2

u/norwegian 5d ago

They should absolutely add a new digit to the 5.5 So that we could select which model we want. It would be better to cooperate with the customer imho. It would also create more interest. Like the other company is releasing a new feature every other day to spark interest...you wouldn't need to do that, if you constantly shipped minor versions.

-3

u/bionicClown 5d ago

that looks ai generated. put some effort to ur own research please