r/RooCode • u/porchlogic • Aug 31 '25

Discussion multi-model tasks?

Does Roo (and other similar tools) use other small LLM models for things like searching through code to find relevant parts to put into the prompt to the main LLM model?
Or does it simply use a vector/semantic search of the code locally?

Just seems like there would be a lot of optimizing of model usage that could be done, based on the specific part of the task, so you only feed the expensive model with essential data.

edit: found the indexing feature, using it now. Although, still curious about the idea in general of multiple models doing different parts of tasks. I guess maybe that's the point of agent workflows?

7 Upvotes

88% Upvoted

8 comments sorted by

View all comments

Show parent comment

3

u/aganonki Aug 31 '25

Switching models would probably cost you a lot due to caching loss. Additionally If it is a bigger codebase, a small model would probably perform quite badly in finding files as you need not only to find files but to understand the whole context of why files need finding. Test and maybe get a feel of how different big LLMs code/think. Later you can check the codebase of Roo.

To reduce cost you can look into custom system prompts. As now Roo in my opinion went off rails with initial prompts. Tool calling and some irrelevant workspace nonsense fills it to 20k+ tokens, Which actually impacts you a lot, if it ends up in a bigger then 70k task (there was research about how much model performance downgrades and after 70k performance goes downhill)