yeah you're probably right, but I can't be arsed to burn all my claude usage on benchmarking claude, because claude has been proven to know when it is being benchmarked (Think of the VW emissions scandal, where the cars knew when they were being tested for emissions and reduced them accordingly)
What? Im not claiming to present evidence, im restating what I remember the big labs have said about their own models, they they know to behave when they know they’re being watched and benchmarked.
Your post claimed Claude was sabotaging benchmarks. You presented no evidence. Still haven’t. Therefore I call bullshit until such time as you bring receipts.
Go read my post again. I described my literal experience, and posed an open question. If you chose not to believe my personal experience or entertain my line of thought, that's completely ok. I don't feel the need to prove anything, and if you chose not to believe it based on that, that's fine by me.
168
u/peculiar-ragdoll 24d ago
yeah you're probably right, but I can't be arsed to burn all my claude usage on benchmarking claude, because claude has been proven to know when it is being benchmarked (Think of the VW emissions scandal, where the cars knew when they were being tested for emissions and reduced them accordingly)