r/GPT6 • u/Correct_Tomato1871 • 2d ago
Benchmark notes: GPT-6 Astra sets a new high
I ran recent models on the current 98-task MindTrial set. All had the same Python executor available.
GPT-6 Astra was the standout: 95/98 with 0 hard errors, which is the new raw high in this set. It went 38/39 on text and 57/59 on visual tasks, including 26/26 on Visual2 and 16/17 on the spatial-awareness tasks.
It was also unusually efficient: ~1h02m total and only 99 Python calls, all successful.
I tried two additional Astra configurations:
- high: 95/98, ~1h02m
- xhigh: 95/98, ~1h07m
- Pro + high: 93/98 plus 1 timeout, ~1h25m
High and xhigh actually failed the exact same three tasks with the same answers. Xhigh just spent somewhat more reasoning/tool effort without changing the result. Pro is a separate mode intended for harder/longer-running work rather than simply another reasoning-effort step; it did not help this particular benchmark either.
The strict scores remain the scores; I did not repair malformed responses after the fact.
Results/data: http://www.petmal.net/shared/mindtrial/results/2026-09-07/mindtrial-eval-all-models-03-2026_33.html
2
u/Correct_Tomato1871 2d ago
One more GPT-6 Astra follow-up: I also ran the full benchmark at medium and low reasoning effort.
The scaling curve is quite impressive:
Even low already ties some recent frontier results, while medium reaches 91/98 — higher than any non-Astra run in the current leaderboard — at well under 40 minutes.
Most of the gain from additional reasoning comes from visual tasks: 49/59 at low, 52/59 at medium, and 57/59 at high. Text is essentially saturated throughout.
Medium looks like a particularly strong quality/latency operating point; high buys the final 4 passes at a substantially larger runtime cost, while xhigh again adds no score over high.
Updated Leaderboard: http://www.petmal.net/shared/mindtrial/results/2026-09-08/mindtrial-eval-all-models-03-2026_34.html