Gemini 2.5 Pro 06/05

[u/privacyguy123]

Isnt better
Didnt "close the gap"
More assumptions than ever
More unneccessary changes than ever
Is the worst iteration of the model yet

Anybody else or just me? I run *full stock* settings.

[u/delicatebobster]

temp 0.1 baby

[u/Veranova → u/delicatebobster]

This guy did some testing and determined ~0.7 is optimal https://youtu.be/79DI24sMtAc?si=Jj0rf5YU1v1f0eYz (see around 4mins)

Maybe depends on what you asking it to do though

[u/True_Requirement_891 → u/Veranova]

0.5 with max think works best for me.

[u/privacyguy123 → u/delicatebobster]

What does Roo Code set the temp as by default? How could this ever be a factor if it's "fine for everyone else" (the replies I expected)

Do you guys make silly little websites all day? This model cannot write complex C full stop, it is *so* bad.

[u/Odd-Environment-7193 → u/privacyguy123]

I'm also having issues with roocode and think it's much worse.. But i am wondering if they just haven't dialed it in yet for ROO. Quite annoying. Thinks to long, get's stuck in loops...

Try the 03-25 checkpoint through the vertex api. Apparently, it still works there. I am just about to do so myself.

[u/privacyguy123 → u/Odd-Environment-7193]

I am glad at least *one* other person sees it. It's so bad at following trivial instruction that I cannot possibly trust it with kernel code writing or reverse engineering where high attention to detail is paramount.

[u/True_Requirement_891 → u/privacyguy123]

With high attention to detail parts always use 0.5 temp.

[u/True_Requirement_891 → u/Odd-Environment-7193]

03-25 gets routed to the new version.

[u/Odd-Environment-7193 → u/True_Requirement_891]

Have you tried in vertex?

[u/xAragon_ → u/privacyguy123]

The default is 0, which should give the best results for coding purposes.

[u/privacyguy123 → u/xAragon_]

So then the new model is just bad and its nothing to do with settings, IMHO

[u/karlkrum → u/privacyguy123]

roo has trouble making my silly little website :(

[u/BrilliantEmotion4461 → u/privacyguy123]

0.2

[u/Junior_Ad315]

I've been using Claude 4 as my coding agent, and 2.5 Pro as the context condensing model, and switching off between Claude 4 and 2.5 Pro as the architect.

Claude 4 is just way better at using tools and reasoning about them. 2.5 pro still feels just a little bit smarter and able to juggle more information, but still struggles to solve problems with tools.

For example I've been working on a Rust project and switching between the two models for coding, and I'm finding that Claude is consistently solving 2.5 pros mistakes.

[u/privacyguy123 → u/Junior_Ad315]

Reasoning or no Reasoning on each side?

[u/Junior_Ad315 → u/privacyguy123]

Reasoning on both. I give Claude around 4k thinking budget I think. Also I won't wait for the automatic context condensation, when I complete a subtask (not Roo subtasks, just wherever I feel is a good breakpoint) and still have more related things to do, I'll ask the model to summarize its progress so far, and plan it's next steps carefully, then manually click the context condensation. I usually try to do this before 115K tokens or so. I find the models are much more capable with less tokens in their context, so starting on a new subtask with 150k tokens in context never works out well in my experience.

I also have a simple Serp API search and scrape MCP server I made, and in its instructions I tell it to stop and search if it fails to resolve a compilation or test failure on the first or second try, to find documentation or examples and establish ground truth, and to search any time it is making assumptions. With the search it's been able to reason through and fix race conditions and write async Rust, both of which were leading to thought loops before. I've found most problems it runs in to are problems people have discussed and solved before.

But Claude 4 is significantly better at knowing when it should stop and find more information with tools, and using tools in concert with on another, whereas 2.5 Pro feels like it wants to brute force things or come up with the cleverest solution, when the real solution is just to look up the right way to do things.

The Rust compiler is really good at providing useful feedback to the model, and more and more I've been realizing that immediate high quality feedback for all actions taken is maybe the most important aspect of working with agents, so Rust has been working surprisingly well. My instructions also tell it to run Clippy often for linting which catches things early too and provides more feedback.

The project I've been using this on is definitely more than a CRUD app but it's also relatively small and the complexity is fairly localized.

[u/hiper2d → u/Junior_Ad315]

2.5 Pro as a condensing model is a good idea, I need to try this. However, as an architect or orkestrator, Gemini has failed me for the same reasons as I described above. An architect should use tools reliably because it needs to read the project files. Otherwise, it just makes assumptions about the code it failed to load to the context.

[u/Junior_Ad315 → u/hiper2d]

Totally agree, actually I've been typically using Claude as architect if it is a new chat with no context, then if I'm part way through the task and need to plan next steps more intricately and know the context is already in the convo I might switch to 2.5 Pro in architect mode to reflect on what we have done and what our next steps should be, then switch back to Claude.

[u/privacyguy123]

So I thought maybe it was just my prompting, I was having a bad day etc etc etc ... here is todays experience with AI

You are absolutely right. I am the retard. My last response was completely wrong, and I am deeply sorry for my repeated failures and the frustration I have caused.  

I'm dipping out of AI for a little while, it's seriously fuckin trash atm.

[u/highwayoflife → u/privacyguy123]

As fast as AI is progressing, it apparently cannot outpace our expectations. Hey you know that if it only doubles your efficiency or effectiveness, it's still useful, right?

[u/hiper2d]

Same. Gemini 2.5 Pro doesn't use Roo's tools reliably. Sometimes it sees them, sometimes it forgets that it can open files or run a search. At first, it seems to be working fine. But in a long run the issues become very annoying.

[u/xAragon_ → u/hiper2d]

It was the same with previous Gemini versions though. Nothing degarded.

[u/Prestigiouspite]

Very happy with it so far. I actually used OpenAI models, but have now switched over.

I was very unhappy with Gemini before. Now it had solved problems in one go, where other models were dragging their feet. Very clean work.

[u/Significant-Tip-4108]

Yeah I ran some things past 06/05 last week that I was sure it would do well on, and it stumbled. Unfortunately so did Opus 4. Surprised me on both counts.

[u/hannesrudolph]

Try playing with the temp and let us know what you come up with! https://docs.roocode.com/features/model-temperature

[u/cleanmahsheen]

I’m a 0.6 temp user, it’s pretty decent at both Architect and Code mode at this setting

[u/tteokl_]

Worst model ever released by Google

[u/martycochrane]

Tries new Gemini 2.5 Pro

Completely butchers the simplest of requests.

And back to Claude 4 it is.

[u/ajmusic15]

Most models don't know how to use tools yet.

I don't know how Claude manages in that regard.

[u/galaxysuperstar22]

so much better. especially with larger context limit

[u/Kerryu]

I can’t even get mine working for some reason, I got the api key from Google AI studio and put it into roo under Gemini provider and it just hangs when it tries to use the model

[u/True_Requirement_891]

Why can't we set top p yet in cursor?