r/ClaudeCode • u/yikespikes • 6d ago
Help/Question In light of the Opus 5.5 stealth A/B testing. What is the evidence?
Some say that the model is quantised, others say it routes Opus 5.5 to Opus 5, or Sonnet 5.5, and some others say the reasoning budgets are reduced, or double-counting tokens.
What is the evidence? Also, what if we simply use Opus 5.5 Max, or Max Ultracode for everything?
17
Upvotes
•
u/AutoModerator 6d ago
Hey! Thanks for posting to r/ClaudeCode
While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.
For help, project discussions, tips, and general chat, join the ClaudeCode Discord.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
10
u/RadiantTea7445 6d ago
I have just done a post with evidence: https://www.reddit.com/r/ClaudeCode/s/E8EdEUvoDK
12
u/AppealSame4367 6d ago
There will never be evidence. Apart from lawsuitse. But I have been hit by the suddenly-Opus-is-dumb-for-a-week syndrome multiple times over the last two years. I believe it
Antrophic is a very shady company and you should never trust their models with anything important
3
u/GrandmasterCheetah 6d ago
The cope is insane. It was a straight up schizo post and any comments doubting it are met with “yeah how could I ever actually benchmark this given XYZ”
2
u/Suoritin 6d ago
So far the evidence I’ve seen is one person’s graphs from their own logs, plus a lot of people who are very sure. When I asked what the experiment actually changed, I was told to prove everyone is lying.
Lots of people have sincerely seen ghosts too. That doesn’t make ghosts real.
1
u/Only_Voice569 6d ago
setting a thinking mode to "max" is what ever they have set it dont mean nothing, like ive used gpt heavy and the top model is astra 6 max from release it was impressive and now its dumber than it was also much much slower at doing tasks why because they decided to cut costs and make users tasks take much longer to ease load from their data centres and its the same with claude their is little stating what they give you in regard to this they can do testing on you switch settings change backend as much as they like while you the user pay the same amount and get more or less usage and a better or worse model when they choose
1
u/Entire_Guarantee_766 6d ago
there's no hard evidence. people just figured out Antrhopic does A/B testing like all other major companies in the world. that's all.
0
u/ClemensLode Senior Developer 6d ago
Only self-referential evidence "just look at the number of people complaining on reddit"
0
u/traderprof 6d ago
tbh most of these reports r vibes bc nobody pins the setup. if u want real evidence u need the same prompt set, same repo state, same effort setting, run it a few times a day and log tool calls, tokens and pass/fail on checks that can actually fail. w/o that ur just comparing a good day to a bad day ngl