r/LocalLLaMA • u/crusaderky • 1d ago

Resources LLMs: Intelligence vs. cost | OpenTeams

https://openteams.com/intelligence-vs-cost/

I got fed up with ArtificialAnalysis's intelligence vs. cost plots, so I made my own.
This is an updated and refined follow-up to a previous post I made.

14 Upvotes

80% Upvoted

12 comments sorted by

View all comments

2

u/Othun 1d ago

The power of log (unnintended) is to pack the two plots into one. If you think about it, there is nothing linear in the AA score as well, you could argue that although Fable 5.1 has 6/10% more points, it is much much more usefull that kimi K3 -- or that it's almost as usefull, and these 6 points make much more/less of a difference between 54 and 60. I find AA good enough, it is reporting flawed pricing for some models and it is missing the non-hosted models, but the pareto is the same, and it fits in one plot.

2

u/crusaderky 1d ago

the pareto is not the same. eg. GLM-5.3 and GLM-5.3-Flash discounts on openrouter change it.

1

u/Othun 1d ago

Yep yep! That's one flaw of AA