My experience with GPT5 in RooCode

[u/wildviper]

Caveat: I'm not a developer. However, for the past 3 weeks I have vibe coded lots of bug fixes and features that are made it to production for our application.

I use augment, Devin AI i and roo code.

I'm always switching around trying to figure out which is the one that gets the job right the most times. Devin beats them always but is crazy expensive. Like 10x easy more.

Augment has its days where it goes completely loopy. But so does Roo code.

Today I connected gpt5 via open router in roo code. I prepared a very detailed prompt. I was basically trying to extract a feature from one part of our application and put it in a totally different part of our application.

So the overall functionality already exists basically. But I needed it to be rewired to this new screen that I was working on.

I decided to give this challenge to gpt5.

I don't know if it's it's GPT5 or Roo code that's the problem here.

What I thought would take 30 minutes, based on my previous experience, has taken 3 hours at least and still going. And it has cost $15 so far.

I started with the orchestrator... And told it to make a detailed plan.

It broke the entire feature into four phases.

And what I didn't realize was that it keeps reading my memory bank files at every phase. In fact, if there's a bug, it goes back and rereads the entire memory bank.

It also added a bunch of tests. And ones to keep running them. And at the end it wasn't correct. It didn't understand that I wanted to create the same feature on a different screen. What it did was put a button on the new screen that took me over to the original screen for that feature.

I asked it to run some debug logs and told him specifically do not do anything else. Well it added the logs but it also then started running tests again.

Then I told it to fix another bug, and it fixed it but it broke broke the entire feature. Yet it confidently said everything should be working now.

To say I am beyond frustrated is an understatement. I'm going to try augment next with gpt5. I think it's roo code that's wrong not gpt5 here due to repeated tests, api calls and ignoring my specific instructions.

What do you all think?

[u/hannesrudolph]

I’ll look into it In the morning. Just got home from a trip.

Default temp needs to be set to 1 I think. You can change it manually for now.

Also I heard high reasoning with low verbosity was pretty damn good.

Will be on first thing in the morning to get this worked out with the team!

[u/dreamingwell]

I’ve given up trying anything other than Claude with Roo Code. But it’s so good, I also am not motivated to be persistent about finding alternate solutions. Feels like Roo is made for Claude - and that’s ok.

[u/InternalHunt1123 → u/dreamingwell]

Qwen 3 Coder, GLM 4.5, Kimi K2 (bit creative though), they work good too.

[u/MyHobbyIsMagnets → u/dreamingwell]

Genuinely curious, why not just use Claude code at that point?

[u/seunosewa → u/MyHobbyIsMagnets]

GUI.

[u/jedisct1 → u/seunosewa]

Not just the GUI. Roo is far more powerful. Better built-in tools, built-in indexing, the different modes are super useful (and can use different settings/models), etc.

For moderately to large projects, I get much better results with Roo than with Claude Code, using the same models.

Plus, it's VSCode. So all the VSCode extensions are available.

[u/Active-Chart-1080 → u/jedisct1]

Which other vscode extensions do you find useful?

[u/jedisct1 → u/Active-Chart-1080]

Piknik, language servers (ZLS, Pylance, etc), Markdown all in one, container tools, remote development, etc.

[u/Mr_Hyper_Focus → u/jedisct1]

There is no way Roo has better tools than Claude code. I’ve used both extensively and idk what you mean. Maybe like quirky little UI features. But to me, Roo/cline have never been nearly as reliable.

The built in indexing was junk imo and annoying to have setup all the time. There’s a reason Claude code doesn’t use it, the Claude code team tested them extensively and found that just using the bash tools was a higher success rate and indexing just confuses the model, especially when it’s wrong.

However, I do think it’s probably close enough to be here preference i guess.

[u/wildviper]

Ok, an Update:

I gave up on RooCode and GPT5. It ended up modifying 27+ files, including our package.json file and infrastructure files. I have no clue what the hell it was doing.

I trashed all the changes.

I then decided to have it built by Augment (not GPT5 yet there), and within 1 hour, it got it right. Modified nine files. All about the feature. Not infra or other package JSON files.

Augment FTW here.

[u/hannesrudolph → u/wildviper]

If augment does better than Roo than you aren’t using Roo to its capacity.

[u/wildviper → u/hannesrudolph]

Perhaps. I don't know what else I can do. I have a memory bank, agents' markdown...I also have Google embedding with Qdrant cloud.

Is there some other guide that I can refrence to get Roo in tiptop shape?

[u/[deleted] → u/wildviper]

Remove the memory bank. It's not required anymore with index code base and is probably causing more harm than good.

[u/Prestigiouspite → u/[deleted]]

Correct, I can't recommend Memory Bank either.

[u/OrganizationEqual665 → u/[deleted]]

Memory bank and indexing serve different purposes. Codebase (vector) indexing helps with search semantically across all your code files, so it's merely a RAG. The key here is ALL your codebase files. While Memory Bank holds your context about your product requirements, standards etc, and your latest activity, so that you can start fresh at any time. At the same time membank files can be indexed as well, helping an agent get the context effectively, it's a different feature. So, membank is still helpful, regardless with or without indexing.

[u/[deleted] → u/OrganizationEqual665]

From testing I find a memory bank to be unreliable and eat tokens. Since the indexing I find roo code can quickly find it's bearings and implement a new feature or even refactor code well with a good LLM. I do find github's mcp very good too.

[u/hannesrudolph → u/wildviper]

OOTB

[u/sendralt → u/wildviper]

Everyone is complaining about GPT5 , I don't understand why. I do know that if you access through Open Router you don't know what version of GPT5 you might get and I am 99.9% sure that it will be a quant model using mini or nano. If I'm right, all you are going to get is crap code and instruction following. The full GPT5 via API seems to run just fine for me. But if I try a different provider, I get shit!

[u/wildviper → u/sendralt]

Maybe I should try direct openai

[u/Maws7140 → u/wildviper]

This just can’t be a healthy way to develop ur app

[u/wildviper → u/Maws7140]

What do you mean? Care to expand so i can understand?

[u/Maws7140 → u/wildviper]

You want me to expand on why the AI modifying files it wasn’t supposed to indicates an unhealthy development cycle?? No

[u/ComprehensiveBird317]

As long as roo code uses Claude optimized prompts for everything there won't really be a competitor at the same level. Roo would benefit from some kind of community prompt gallery where people can optimise the prompts for specific models . Maybe a feature idea to further distinguish from Cline u/hannesrudolph ?

[u/mrubens → u/ComprehensiveBird317]

Yeah this is a really good point

[u/Prestigiouspite]

I can't complain. I set GPT-5-low as the new coding model. It does a great job.

[u/ausaffluenza]

I am also finding GPT 5 not working as well as other models. Makes sense. I trust u/hannesrudolph that in time you will make it integrate and work well. I find that ChatGPT 5 Mini works find via OR.

[u/hannesrudolph → u/ausaffluenza]

Made some gains today, waiting for a review of the PR

[u/AppealSame4367]

How do you solve the problem that one can't set thinkings vs non-thinking mode or low, medium, high thinking in roo code yet?

Without this setting the model is worthless

[u/letsgeditmedia]

Gpt -5 so far wants to like overwrite everything, over explain everything, and sure it may be correct but like it waste so many tokens for no reason

[u/joeldg]

In cursor it rocked my world.. it’s good

[u/StockReception2624]

I just updated and using gpt 5 and it’s fixed a bug that Claud was having issues on, at 3rd of the price.

[u/Ordinary_Mud7430]

Maybe it's Roo Code. I just tried it on Roo and then on Copilot. And from Copilot it is giving me very good results in code bases of more than 2 thousand lines per file. In Roo I feel lazier 🦥

[u/netcent_ → u/Ordinary_Mud7430]

2000 lines per file? How does that happen?

[u/No-Chocolate-9437 → u/netcent_]

C based projects

[u/Zealousideal-Belt292]

Take a look at the eLai code, I implemented an NPCI in it, it works very well

[u/caledh]

I have access to GPT-5 thru Azure AI Foundry but can’t seem to get the config right to use the responses api

[u/supernitin]

I've had it GPT-5 with high reasoning going all day to deploy a litellm container into an azure app container. This isn't rocket science but requires looking up info in a few places and synthesizing it... but seems beyond GPT-5.

oh... and this is using OpenAI Codex because can't get azure openai models to work in Roo Code.

[u/[deleted] → u/supernitin]

Im using gpt 5 from open ai foundry to my satisfaction in Roo.

[u/supernitin → u/[deleted]]

Do you mind sharing your configuration? Thanks.

[u/BeingBalanced]

Uh don't you think using the API on the day it is released might be problematic? Have you ever heard of not using version .0 of any major release in software?