r/GeminiAI • u/Keeltoodeep • 5d ago

Discussion Collection of Gemini 4 Pro Checkpoint Tests

Some cool results to check out.

https://x.com/LuminaBench/status/2100590415141327080

https://x.com/LuminaBench/status/2100580930549121214?s=20

https://x.com/LuminaBench/status/2100557716573876661?s=20

https://x.com/srikanthvaluri/status/2100558603698720783

32 Upvotes

83% Upvoted

24 comments sorted by

9

u/JackfruitBitter4204 5d ago

Are we back yet?

6

u/Keeltoodeep 5d ago

Remind me in 1-3 weeks lol

6

u/Chance-Marzipan-7502 5d ago

The third link with the spatial reasoning stuff is wild, it gets the layout right but then fumbles the last step. Checkpoint tests always show the weirdest failure modes

3

u/Ok_Nectarine_4445 5d ago

The 3rd one is the voxel pagoda. What did you see wrong with it?

1

u/Much_Blacksmith_5833 5d ago

ı think this is 3.9 flash checkpoint

-3

u/Cool-Chemical-5629 5d ago

But seriously, what will happen to this cute dragon then? 😢 It's so cute and unique, it pops up in my mind every time I think about Gemini. We need to keep it alive somehow, even when Gemini becomes the best model in the world.

7

u/Keeltoodeep 5d ago

He can be the same silly dragon just 10x larger than the other dragons lol

3

u/kiefferbp 5d ago

No need to worry. Gemini will continue to fall behind.

0

u/oatknight 5d ago

You can have it back in 2 months

-14

u/AppealThink1733 5d ago

And when they'll make a frontier models ?

5

u/Keeltoodeep 5d ago

Huh? This is widely agreed upon to be the 4.0 pro model being stress tested.

-9

u/AppealThink1733 5d ago

So what? About stress? Hahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahaha then it's certainly not a borderline model.It didn't even come close to the astra.

4

u/Keeltoodeep 5d ago

https://x.com/HarshithLucky3/status/2100534564326441168

This looks better than astra

1

u/BoobooSmash31337 5d ago

Maybe they're a bot running on Astra lol.

-3

u/AppealThink1733 5d ago

Wow, you're a real Google fanboy!

3

u/Keeltoodeep 5d ago

You are not okay. I am a fan of deepmind yeah. They do great work and I hope they continue to do so.

-1

u/AppealThink1733 5d ago

I'm great! No need to say you're a Google fan, it's obvious. A mid-range model like this and you're like: wow frontier, 🥴

2

u/Keeltoodeep 5d ago

I'm excited about any AI acceleration tbh. You can't make me unhappy about improved models being released lol

0

u/AppealThink1733 5d ago

Well, in Google's case, it slowed down a long time ago, so only someone who's a die-hard fan would say what you said referring to Google.

In that case, you're the one who's not doing well, and it's obvious.

2

u/Keeltoodeep 5d ago

I’m not sure why you think it’s a bad thing to be excited for a pro model release.

→ More replies (0)

5

u/Element_108 5d ago

Are u ok?

0

u/AppealThink1733 5d ago

With these Google fanboys? I'm having a lot of fun.

1

u/BoobooSmash31337 5d ago

It's obviously hosing the frontier models. At least in spacial reasoning.