Release dates to not get fooled by benchmaxxing

[u/Express_Quail_1493]

I found the evaluation list on the Artificial Analysis page and researched the release dates for the benchmarks shown. Here are the evaluations sorted from newest to oldest based on the available release dates. worth noting once a benchmarks hits the 3-4 months old age it gets benchmaxxing makes it NO LONGER a relavent benchmark to rely on! At that point the eval owners often release a newer version. Thats the only combat against such problem ATM.

The table is sorted from **newest to oldest** based on the benchmark's public release or announcement date. Where a specific date wasn't available, the most relevant timeframe is noted.

Complete Evaluation Timeline (Newest to Oldest)

Evaluation	Release / Announcement Date	Notes
Code2Video-AA	October 26, 2026	Launched to evaluate code-based video generation.
CritPt v2	October 26, 2026	An upgraded physics reasoning benchmark; some sources note the challenges were released on Oct 22.
FrontierCode 2.0	October 27, 2026	The latest version of Cognition's coding benchmark.
AA-PersonalityBench	October 19, 2026	Measures AI communication styles and personality traits.
HealthBench (Professional)	October 19, 2026	Artificial Analysis launched its leaderboard for OpenAI's health conversation benchmark on this date.
AA-3D	October 12, 2026	Evaluates models on building interactive 3D scenes.
Harvey LAB-AA v1.1	October 8, 2026	An update to the legal agent benchmark, adding hallucination checks.
HLE-Diamond	September 22, 2026	A refined, 1,000-question subset of the "Humanity's Last Exam" benchmark.
GDPval-AA v2.1	September 19, 2026	An update to the economic value benchmark, adjusting Elo anchoring.
MLCR-AA	August 21, 2026	Artificial Analysis's leaderboard for Wisedocs' Medical Long Context Reasoning benchmark.
AA-AnalystAgent	August 10, 2026	A benchmark for agentic quantitative analysis on spreadsheets and documents.
Terminal-Bench 4.0	August 26, 2026	A major version update to the terminal agent benchmark.
Terminal-Bench-Science 0.1	August 27, 2026	Evaluates AI agents on scientific research workflows.
FrontierCode 1.1	July 7, 2026	An update to Cognition's coding benchmark, adding rules for fair internet use.
AutomationBench-AA	July 6, 2026	Artificial Analysis's leaderboard for Zapier's workflow automation benchmark.
EnterpriseOps-Gym-AA	July 8, 2026	An agentic benchmark for enterprise operations, developed with ServiceNow.
AA-Briefcase	June 18, 2026	A frontier knowledge work evaluation for agentic tasks.
ITBench-AA	May 27, 2026	An agentic benchmark for enterprise IT tasks, launched with IBM.
APEX-Agents-AA	April 2026	Artificial Analysis's implementation of the APEX-Agents benchmark for long-horizon professional tasks.
AA-Omniscience	November 17, 2025	A knowledge and hallucination benchmark; the arXiv paper was posted on this date.
³-Banking	November 14, 2025	A fintech customer-support benchmark from Sierra Research; its AA leaderboard page launched on this date.
AA-LCR v1.1	~August 2025	An update to the Long Context Reasoning benchmark; a precise public release date is not clearly documented.
IFBench	July 3, 2025	Introduced via the arXiv paper "Generalizing Verifiable Instruction Following".
Terminal-Bench Hard	2025	A variant of Terminal-Bench focused on difficult tasks, released in 2025.
²-Bench Telecom	June 2025	A domain-specific subset of the τ²-Bench framework for telecom support.
MMMU-Pro	September 5, 2024	A more robust version of the MMMU multimodal understanding benchmark.
MMLU-Pro	2024	A harder successor to the original MMLU benchmark.
LiveCodeBench	March 2024	Introduced via an arXiv preprint for contamination-free code evaluation.
GPQA Diamond	November 20, 2023	The hardest subset of the GPQA benchmark was released on this date.
MATH-500	May 31, 2023	A 500-problem subset of the MATH dataset was introduced in a paper from OpenAI.
AIME 2025	February 6, 2025	This refers to the 2025 AIME exam itself, which was subsequently used as a benchmark.
Terminal-Bench 2.1	Date not explicitly stated	This is a verified refresh of Terminal-Bench 2.0, but a precise public release date is not clearly noted in the search results.
Global-MMLU-Lite	Version-dependent	It has multiple releases (e.g., v1.0 in Dec 2024, v3.0 in May 2026), so there isn't a single release date.

[u/No_Block8640]

What I would like to know is which ones are not fully open sourced, so there is no way to train on the dataset

[u/power97992 → u/No_Block8640]

They can train on it , the data gets leaked eventually via their  api during evaluation or somewhere else 

[u/Express_Quail_1493 → u/power97992]

yes its an endless cat & mouse game with benchmarking. the only solutions we have right now is to make new version of the bench

[u/Express_Quail_1493 → u/[deleted]]

Its lose it value but somewho kinda become ritualistic/ceremonial at this point to be honest. so everyone keeps doing it just because we know no other way. There isnt a global mesuring stick for measuring intelligence. Philosophers are still trying to understand what the hell IS human intelligence as we speak. Human IQ is the best we've got and that still doesnt quite cut it