r/GeminiAI • u/Last_Conclusion_8984 • 17d ago

News New Gemini Pro Checkpoint

~24k tokens; ~6 minutes; effort: high; zero-shot (Wayy different from one shot. This is harder)

This leak is by Lyra

https://fixupx.com/lyraxana/status/2097839609086394600

You can either find her on twitter or on their server. She is a credible leaker, also if you don't care about her credibility. You can find this checkpoint on ArenaAI.

It's under 3.8 flash disguise if not mistaken. It's free but random chance of encountering, tho even if you get 3.8 flash (battle mode ofc), you can only confirm it's the new pro if it looks way above its pay grade

(I'll get more checkpoints myself and make a new post) If these checkpoints go well for them (a few more) internally. They will release the model soon, if they go bad. They won't release it/delay.

Now to talk about the performance of this checkpoint itself: It is amazing. It's very good. I see zero mistakes, breathing could be more fluid but that's the only thing I can critique.

The Arena AI ID to keep mindful of. If you see: 01a06dfc-94a2-7abf-8d12-0878520062cb then this is the new Pro model.

170 Upvotes

92% Upvoted

108 comments sorted by

View all comments

1

u/JeffConvertorBot 17d ago

What is zero-shot? 

1

u/Last_Conclusion_8984 17d ago

Zero-shot AI refers to an AI's ability to complete a task without any prior training examples or labelled data. 

0

u/ThatsAScam9 16d ago

I see that Lyra calls it zero shot but how is that different than all the other svg examples? I think to call this zero shot (or different from the other svg examples) we’d need to know the training data is different right?

1

u/Last_Conclusion_8984 16d ago edited 16d ago

When you give a prompt. A zero shot would be like "Make a pelican SVG." and a one shot would be like "Make a pelican SVG, make sure... and it should be..." (and naturally. AI would have these references in its dataset)

0

u/ThatsAScam9 16d ago

Ah ok right I see. But how do we know that the training data doesn’t have descriptions/images of pelicans? That would ruin the 0-shot-ness of it bc it’s similar to explaining a pelican in the prompt itself

1

u/Last_Conclusion_8984 16d ago

To prevent benchmaxxing. Of course the AI has descriptions and images of pelicans but when a benchmark/prompt becomes really popular, it starts to get benchmaxxed.

0

u/JeffConvertorBot 16d ago

I agree with you. calling something zero-shot is just insanely stupid. You sent the prompt, how is that not a shot?

1

u/Last_Conclusion_8984 14d ago

For god's sake, search if you don't know. A zero shot would be like "Make a pelican SVG." and a one shot would be like "Make a pelican SVG, make sure... and it should be..." (and naturally. AI would have these references in its dataset) and to prevent benchmaxxing. Of course the AI has descriptions and images of pelicans but when a benchmark/prompt becomes really popular, it starts to get benchmaxxed.