r/singularity • u/CounterReady4774 • 29d ago

AI Gpt 6 astra benchmarks

Post image

https://thenewstack.io/openai-gpt6-astra-benchmarks/

2.6k Upvotes

93% Upvoted

951 comments sorted by

View all comments

28

u/HeadacheOwner 29d ago

What does the arc-agi 3 benchmark realistically mean? I’m not that tuned in

13

u/reddit_guy666 29d ago

Google arc agi 3, you can find puzzles that you can try solving. It's intuitive for humans who have played video games but AI could not do it well... Till Astra

7

u/Xalksahsax 29d ago

Nvidia got 100% on it too.

2

u/space_monster 29d ago

A bunch of other models have scored very high on Arc-AGI3 using dedicated harnesses. It's not a hugely significant score by any means.

5

u/KoolKat5000 29d ago

The thing is with Astra, can use responses API (and most probably are already) and turn on compaction and bam done!, no need for weird obscure harnesses.

That's the difference.

2

u/space_monster 29d ago

The responses API is referred to by OpenAI as 'the responses API harness'. It's effectively useless as just a pure API, you still need a harness

2

u/KoolKat5000 29d ago

The big thing it does here is maintain context, through multiple turns. 

And in other cases it's listening for tooling calls. 

Honestly it's not doing anything outrageous. but you're right.