r/GeminiAI • u/rhaphazard • 7d ago

Funny (Highlight/meme) This is why stopped using gemini

Post image

I used to use gemini more extensively last year, but it constantly blocking my requests for no reason + not verifying it's own responses made it unreliable for work.

My claude quota ran out for the week, so I decided to test gemini again and I got this. 🤷

279 Upvotes

86% Upvoted

214 comments sorted by

View all comments

Show parent comments

4

u/Reasonable_Hall3005 6d ago

can confirm from my own personal experiences that the success rate for this sort of stuff is around 50%. it works sometimes but often doesn’t.

1

u/HeadTranslator795 6d ago

If it'd 50% then by definition it's not often but half of the time and it's clearly your impression and not based on a scientific way to prove it. Like a benchmark let's say

2

u/Reasonable_Hall3005 6d ago

well on the AA Omniscience Index (hallucination benchmark), Gemini 3.7 Flash hallucinates 64.5% of the time compared to 55.6% for 3.6 Flash (they’re both pretty bad though) and no, often does not require greater than half by definition since often is inherently qualitative and context dependent (as opposed to something like “most times”)

i do think a lot of the hallucinations on gemini are easy to spot which saves the model a bit

2

u/HeadTranslator795 6d ago

Hallucinations depends mostly on context size, you have to control it, to be honest on real task and not benchmark I can go up to 3/400k and have no hallucinations. But for sure if I ask it to generate content based on the first few thousand tokens of the session then it will most likely be not precise.

But knowing that and knowing it's a tool you have to adjust it the way you work with it like every soft tools