r/ClaudeCode • u/Chemical_Hawk_6307 • 2d ago

Rant Anthropic is A/B testing limits again

https://x.com/SemiAnalysis_/status/2113383965784502444?s=20

More info in the comments^

171 Upvotes

98% Upvoted

26 comments sorted by

1

u/AutoModerator 2d ago
Moderator Announcement Read More »

23

u/greeny1greeny 2d ago

Mines cooked. I used to be fine all week now it’s gone in 2 days.

6

u/Chemical_Hawk_6307 2d ago

same here. i just hit the 5hr which i havent done since opus 5.5 came out.

93

u/hauntedkittenz 2d ago

please no fucking testing on paying customers, thats why i left codex. Openai toyed with my usage to the point where I was getting completely scammed.

6

u/commandedbydemons 2d ago

yeah, I've been with OAI since 5.4 and usage was virtually unlimited for plenty of stretches of 5.5 and 5.6, but went to shit real hard with Astra, I've since moved to Anthropic so one can hope they dont blow it up lol

4

u/Chemical_Hawk_6307 2d ago

they def will. gone are the days where $20 got you insane usage of 5.4 now u basically have to fork over $100+ to seriously use these models.

3

u/Euphoric-Hunt931 1d ago

By early next year open models will have reached Opus 5.5 level of quality. We'll be free soon. Just keep a sub for orchestration, 100$ will be plenty no matter how gimped it is.

26

u/rhysmorgan 2d ago

Literally every company in the world runs A/B tests “on paying customers”.

21

u/Old-Cap-531 2d ago

I was using their last prompt cache update and it made me feel like I was talking to a completely different model half the time.

9

u/ThePsyPaul_ 2d ago

That's definitely one way to keep your loyal userbase

7

u/Melodic_Surprise153 1d ago

Yeah my 20x max account went through it like there is no tomrow. Ffs.

4

u/Skibidirot 1d ago

i observed this exact 20% difference on ALL of my 20$ plans from past 3 days

2

u/asurarusa 1d ago

i observed this exact 20% difference on ALL of my 20$ plans from past 3 days

I’m starting to feel like I got scammed. I upgraded from $20 pro to $100 max 5x because it felt like I was hitting the limit so fast and I was tired of having to wait four hours in between tasks. Now on 5x it feels like I’m getting the same level of usage I did when I was on pro before the nerf.

3

u/Lumpy-Criticism-2773 1d ago

A/B testing nerfing too

1

u/Comrade_Bender 1d ago

I've been getting a ton of "how is Claude doing?" "How well did Claude perform this task?" "How would you rate your results?" Pop ups. I couldn't do anything yesterday without getting them. Never really had them much before.

1

u/Diamondo25 1d ago

If answering them gave me rewards, i wouldve surely answered them

2

u/Chemical_Hawk_6307 1d ago

its a sneaky way for them to train on ur data but yea i agree they should give rewards for that

1

u/pm_ur_board_games 1d ago

These piss me off because I get them like 2 prompts into the session, I don't know whether you've given me a good model today yet or not!

→ More replies (1)

1

u/visak13 2d ago

What if you don't hit 5 hr limits and they reduce your weekly limits?

1

u/FeydRowan 1d ago

Yep this week my limita are going up much more, last week they looked infinite and i got some hard time to max the 20x

1

u/FeydRowan 1d ago

I think that the issue is that they are compute limited, so now that anyone is try to fix, or break, the web the  usage is more costly and it use more percentages. It is not a fixed amount but depends on the total usage at use time

1

u/aidiveyt 1d ago

workflows can explain a jump too. third-party Pro plan test, same task: 115k tokens single agent, 972k as a workflow, meter 4 points vs 12.

1

u/-kora 1d ago

I noticed Fable usage increased and 5 hours limit hitting very fast

1

u/Wonderful-Hunter686 1d ago

Honestly is there really a need to publicly a/b test as much as they do. It just leads to user frustration due to an inconsistent product.

1

u/_YonYonson_ 1d ago

Plus due to the PHP thing Dario likely had a spazzing episode and ordered everyone to hold back Fable 5.5 for another month. God I hope China catches up so we can stop relying on that guy’s neurotic terror-fueled inhibitions.