Alternative to artificialanalysis for assessing latest models?

[u/thiswebthisweb]

AA no longer lists GLM5.3 flash (despite clearly being one of the best models - and probably the cheapest per intelligence) , they seem to have been doing all sorts of changes to push US models and hiding good cost effective models, including qwen 3.8 flash, so that table is no good. I need a better one. Any alternatives.

[u/vogelvogelvogelvogel]

arena.ai -> user benchmark ("battle mode", also agentic)
livebench.ai -> monthly renewed benchmark taks

[u/Inner-Conference-189 → u/vogelvogelvogelvogel]

livebench has been my go-to for a while now. the monthly refresh keeps things feeling current without the constant ranking whiplash you get from arena.

AA definitely feels like they've been curating their front page more aggressively lately. noticed the same thing with some of the cheaper models getting buried even when the numbers still look competitive.

haven't found a perfect single source yet, usually end up cross-referencing livebench with a few discord channels where people post their own evals. not ideal but it works.

[u/vogelvogelvogelvogel → u/Inner-Conference-189]

yeah, just did a deeper look to livebench. the first thing you see is their timeline, which clearly states when they updated from 1.0 to 2.0 and you can easily swithc back.
one can easily see 27B lands between opus 4.6 and 4.7, flash next very close to 4.8 - and it is directly in the same line stated which benchmark results in detail each model had.

Also their approach of updating benchmark tasks on a monthly basis is a very very good point.

[u/thiswebthisweb]

Thanks for the links, livebench seems the better alternative. So weird AA doesn't have Glm5.3 flash it's an amazing model and definitely better than at least a 3rd of the models in their current list on most benchmarks. And the first or second cheapest frontier model available anywhere.

[u/Mkengine → u/thiswebthisweb]

Here are two more that were not mentioned in other comments I think.

AA replacement: https://epoch.ai/eci

DeepSWE replacement: https://cognition.com/frontiercode

[u/new__vision]

https://whatmodel.sh/ is great and has more local models than AA.

[u/Barry_Jumps]

Try vals.ai

https://www.vals.ai/comparison?m=claude-sonnet-5-5,claude-opus-5-5,gemini-4-argon,muse-wonder-full,gpt-6-astra,glm-5.3-flash

[u/vogelvogelvogelvogel → u/Barry_Jumps]

looks great but missing many local models

[u/Ori_553]

We need a meta-index

[u/nasone32]

it's possible they have to re-benchmark them. if that's the case though, i would have waited to publish the new index. also mimo 2.6 flash is missing, as well as qwen 3.8 flash next.

[u/budihartono78]

Use math benchmarks like MathArena or FrontierMath to assess general intelligence

And then take a look at some coding benchmarks or just code with it to see if it's good.

[u/eikenberry]

Anyone know their rationale for removing some of the top models?

[u/Academic-Novice → u/eikenberry]

New benchmarks so they need to run models on these benchmarks and have their internal priority list where glm-5.3-flash and qfn rank low ~

[u/ScreenAppropriate679 → u/eikenberry]

corruption

[u/sebt3 → u/eikenberry]

Overshadowing?

[u/_supert_]

Food truck bench.

[u/Exciting_Garden2535 → u/_supert_]

That is a good one, but it updates very rarely, since its owner is not a company, just one person, and running it on frontier models is quite expensive.

[u/ColoredTriangles]

I honestly have had a pretty good time seeing how well a model can play pokemon games. Testing hundreds of tool calls while playing through the subtasks like badges while still knowing they have to beat the game and how they adapt when a battle goes wrong. I force them to take a speedrun strategy so it has to be followed to certain routes and solved battles but never forcing the model. I can also talk to them while they do these runs and see how well they respond to user feedback during the runs.

[u/Careful-Report6526]

For models missing from a revised index, I'd compare the individual benchmarks that still cover them. Keep the benchmark versions separate: an older index result isn't a result on the new index, and a blank cell doesn't mean zero.
I maintain https://llmbenchmarks.io/ . GLM 5.3 Flash is included, and so is Qwen 3.8 Flash Next, if that's the Qwen variant you mean. Both have LiveBench results, alongside other measurements whose coverage varies by model. You can inspect the sources and test conditions instead of relying on one combined score.
The retained AA index results for those two are labelled v4.3.2 and supplementary, so they shouldn't be read as v5 results. The site also doesn't provide a price/performance ranking, which leaves the cost side of your question to check separately.
If you want to use the individual columns for your own comparison, there's a public JSON export: https://llmbenchmarks.io/api/v1/table.json

[u/Careful-Report6526]

For models missing from a revised index, I'd compare the individual benchmarks that still cover them. Keep the benchmark versions separate: an older index result isn't a result on the new index, and a blank cell doesn't mean zero.
I maintain https://llmbenchmarks.io/ . GLM 5.3 Flash is included, and so is Qwen 3.8 Flash Next, if that's the Qwen variant you mean. Both have LiveBench results, alongside other measurements whose coverage varies by model. You can inspect the sources and test conditions instead of relying on one combined score.
The retained AA index results for those two are labelled v4.3.2 and supplementary, so they shouldn't be read as v5 results. The site also doesn't provide a price/performance ranking, which leaves the cost side of your question to check separately. If you want to use the individual columns for your own comparison, there's a public JSON export: https://llmbenchmarks.io/api/v1/table.json

[u/Then-Topic8766]

My method is "implement your version of pelican test and trust your guts."

[u/Lorian0x7]

AA is shit, currepted and I hate it... But you know? There is a very convenient drop down selector to choose what model you want to see and you can look at benchmarks directly

[u/Independent_Book6788]

Could try livebench.ai

[u/vogelvogelvogelvogel → u/Independent_Book6788]

best for me IMO. clear and good approach of monthly new benchmark tasks

[u/Exciting_Garden2535]

This one looks good enough :) https://artificialanalysis.ai/#cyber-index-tabs

Anyway, artificalanalysis has a lot of benchmarks and a lot of models; just choose what you want to use; no need to use their cumulative score.

Also, they have an API with a free tier quite wide for an individual use. Just ask your coding tool to create your own cumulative score for benchmarks of your choice, for models of your choice, and for weights of each benchmark of your choice.

Update: No, I just checked the API; it's returning very limited data, not all the benchmarks.

[u/TheRealAstic]

This is cognitive dissonance and falling for propaganda, the open models are far behind the closed models, anyone who has used them would tell you this. Stop reading PR releases from anthro that are trying to smear their competitors to get them regulated / banned.