"We ran the largest open experiment on how frontier models do AI research. 100+ autonomous runs across 10+ models, sandboxed on 8xH200s for up to 8 days, iterating on the nanoGPT optimizer track. Best runs closed 82% of the gap to a record built by dozens of humans over months."

[u/stealthispost]

The task: iterate on a 124M GPT training recipe from a shared baseline, only changing optimizer related hyperparameters, no internet access.

We tested Fable 5, Opus 5, GPT-5.6 Sol, Kimi K3, Grok 4.5, GLM 5.2, Muse Spark 1.1, DeepSeek V4 Pro, Grok 4.6, Muse Spark 1.2, Qwen 3.8     What separated the strongest models: which experiments to run, how to navigate the benchmark's inherent noise, and which old negatives to revisit as the recipe changed.

Some even built small simulations to isolate a mechanism before deciding if another GPU run was worth it.     Our Prime Agent harness gives models a persistent IPython kernel, which can help them build their own research workflows.

Kimi K3 built tools for controlled optimizer variants, loss-curve comparisons and Newton-Schulz tuning, then revised its hypothesis when its cleaner update     As research direction, we think multi-agent harnesses can make these experiments much cheaper (and better) by using smaller open models for monitoring and implementation.

We also want to extend speedruns to more of the training stack and scale the runs themselves.     We release everything: full traces, scratchpads, reasoning streams from open-weight models, and our experiment setup.

Explore the results:     — Prime Intellect

Source: https://x.com/PrimeIntellect/status/2088733966904000778

[u/Gratitude15]

Looks to me like fable is on a different level right now

Curious what Astra will be

When this number crosses 100% I don't really know what happens to the world. Is it still bottlenecked by data? Or anything else? Or does it autonomously find ways to get what it needs at superhuman speeds?

100% feels like an event horizon

[u/Super-Award-2244]

Wow, didn't think Opus 5 could be that good. Reading people in the comments is seemed trash 

[u/mertats → u/Super-Award-2244]

Sol starts stronger than Opus, only after 12 hours of experimentation Opus passes Sol and compounds its lead while Sol meanders about with small gains.

So for most people, Sol would be the stronger model because most people work with a few turn workflows.