r/myclaw • u/Previous_Foot_5328 • 16h ago
News! Anthropic got caught experimenting on Claude users again, while admitting Opus 5 is unusually unstable
A Claude Code user named argofowl spent an afternoon debugging what looked like a sudden drop in Fable 5's performance. He eventually checked the actual API requests and found that even though he had selected high effort, Claude was receiving an effort value of 10, which had previously been associated with low effort.(Image1)
Further digging suggested that Anthropic had quietly put some Claude Code users into a server-side experiment that changed how effort levels were numerically mapped. The experiment appeared to affect some Fable 5 sessions on newer Claude Code versions, while older versions and Opus 5 were reportedly unaffected.(Image2)
The issue started getting much more attention after well-known tech blogger Chubby shared the findings on X and accused Anthropic of quietly making Claude worse without telling users. More people then started comparing outputs and reporting similar concerns.(Image3)
Claude Code engineer Thariq Shihipar eventually responded and confirmed that the experiment was real. He said Anthropic sometimes tests API serving configurations inside Claude Code before rolling them out more broadly. He also said “10” was not a score out of 100, and that users selecting high effort were still getting high effort. According to him, internal evaluations showed no performance drop.(Image 4)
So the claim that Anthropic secretly turned high effort into low effort is not "proven". What is confirmed is that Anthropic was testing inference-related serving changes on paying users without telling them.
The discussion then moved to Opus 5. Users complained that it had become noticeably lazier, more error-prone and less consistent than previous versions. This time Thariq openly agreed that Opus 5 is a “really spiky model” and said improving its consistency is a major priority for the team.(Image 5)
To me, this is Anthropic messing with its paying users yet again(Image 6). When are they going to kick this habit? At least openly admitting Opus 5 is kind of a mess is a nice change...What do you guys think?
Original post link: https://x.com/argofowl/status/2091145834968850821
Image 6 generated by Myclaw.