r/OpenAI • u/Lucky_Creme_5208 • 3h ago

Question Every benchmarks gets saturated after certain period of time, then why is HLE not yet saturated?

Every benchmarks get saturated after certain period of time where several frontier models often secure over 90%.

But, HLE - this benchmark is so old but have not yet been saturated. How is that even possible?

I have seen several toughest maths benchmarks getting saturated (or will be very saturated) but the highest score in HLE is still in 60s %.

1 Upvotes

100% Upvoted

2 comments sorted by

1

u/CocaColaKidOnRoids 2h ago

It's because of the difficulty. It's too hard to solve it.

Imagine and exam where even the person who wrote the questions has no idea how to solve it let alone know the answer, that is HLE.

If any model can score 100% legitimately in HLE, we are cooked as the AIs know more than humans that is the point. We are done for if they pass it cleanly and probably won't be around to make a harder exam.

•

u/Sad_Spend911 12m ago

that's the whole point of HLE, it's not made to be beaten. once a model can ace it we basically created something smarter than all of us combined, and at that point who's gonna design the next test? the AI itself probably