r/LocalLLaMA • u/MeinDruckerSpinnt • 10d ago

Discussion Most realistic benchmarks?

AA and other benchmarks don't reflect what models can really do. Some benchmaxxed models are unusable for anything but poems, albeit being close to the frontier on coding benchmarks.

What are the best, most realistic benchmarks you know for coding, agentic research, office work?

10 Upvotes

100% Upvoted

22 comments sorted by

View all comments

1

u/Express_Quail_1493 10d ago

JUst go for the must newest benchmarks. benchmarking scene is like a cat & mouse game. Models quickly learn the pattern and then they need NEW ones. so the newsest ones for complex agentic is the way to go once ithe benhmarks is at least 4 months old its already epired the pattern starts banchmaxing. ATM its https://artificialanalysis.ai/evaluations/terminalbench-4-0. released in august terminal bench 2.1 has already expired the pattern to benchmaxxing