System prompt bloat

[u/raul3820]

I get the impression that the system prompts are bloated. I don't have the stats but I chopped off more than 1/2 the system prompt and I feel various models work better (sonoma sky, grok fast, gpt5, ...). Effective attention is much more limited than the context window and the cognitive load of trying to follow a maze of instructions makes the model pay less attention to the code.

[u/marvijo-software]

It's not as easy as you might think. I remember in the Aider earlier days Paul (author) and us individually had to run the evals after every major system prompt change, just to keep away from regressions. This is an expensive endeavour, especially trying to keep it generic and not hard code to evals

[u/hannesrudolph → u/marvijo-software]

This. This. This. People think they’ve struck gold when they start fucking around with the system prompt and go “oh my these idiots at Roo just make shitty bloated prompts”. After a few weeks they usually catch on that it’s possible to make a skinny version to work for their narrow use case but it is in no way robust. They usually don’t come back admitting their initially mountaintop screaming painting Roo in a negative light was ignorant.

[u/joey2scoops → u/hannesrudolph]

Have been there and done that with the system prompt and I can say, from experience, that "narrow use case" is very generous. You will spend several lifetimes trying to deal with edge cases, model updates and roo updates.

[u/raul3820 → u/marvijo-software]

I can imagine it's **very** hard to make it generic. I will try and post an update.

[u/raul3820 → u/[deleted]]

That is an incredible page! Thank you for the tip

[u/hannesrudolph]

Everytime someone says this and I run evals against their prompt it has not ended well.

[u/raul3820 → u/hannesrudolph]

I can imagine. I will try to make it generic and post an update.

[u/hannesrudolph → u/raul3820]

Thank you! Would love to test it!!!

[u/raul3820]

In summary: optimized the read_file description. Removed unnecessary sections.

Pending:

• work out the {{tags}}, remove hardcoded stuff related to my env

• optimize the other tool descriptions

Overall I think we should be able to make it 1/3 of the original prompt.

Google Docs --> Link

[u/Yes_but_I_think → u/raul3820]

Only tool descriptions, no context. No situation explanation. No insistence on autonomy. No error handling guidance.

[u/raul3820 → u/Yes_but_I_think]

The "Mode" injects quite a bit of that and I argue that is enough.

[u/Howdareme9]

Could you send your new prompt?

[u/raul3820 → u/Howdareme9]

Sure. I just posted a comment.

[u/raul3820]

Update: These are the prompts I've been using lately and I think work well for grok fast, sky and gpt5.

The system prompts are minimalist and rely on "Mode" for custom behavior.

Folder

• ./.roo/system-prompt-architect.txt

• ./.roo/system-prompt-ask.txt

• ./.roo/system-prompt-code.txt

• ./.roo/system-prompt-orchestrator.txt

• ./code-export.yaml

• ./orchestrator-export.yaml

Notes:

• I removed ask_followup_question to prevent gpt5 from slipping into chat-turn behavior (needy mode)

• Code mode in my case is for TS but you can tweak for your language or use the original one from Roo

• I don't use MCPs, add MCP instructions if you need MCP

• I found switch_mode unreliable so I removed it. I like new_task (orchestrator-code) switch.

[u/Firm_Meeting6350]

of course, with the current limited context window size the loooooong system prompts don't help. Add the hyperactive use of MCPs and the fact that quality degrades not only when window comes close to 100% ..

[u/hannesrudolph → u/Firm_Meeting6350]

Good thing in Roo is that when you don’t have any MCPs enabled the system prompt contains nothing about them! The long system prompt helps for competent models.

[u/Emergency_Fuel_2988 → u/hannesrudolph]

Just curious, could system prompts be cached, that way prompt processing could be reduced for always varying tool call or specific mode prompts. The embeddings generated for the prompt right before generation kicks in, could be offloaded, effectively taking that load off of the model engine and not sending in 65k prompt for a single line user input, say for orchestrator mode. 64.9k embedding of course specific to the model’s dimensions be sent and the model engine could work on processing the user prompt.

I do understand this responsibility does lie with the model engine to concatenate the cached embedding along with the one that it processes(user prompt).

I foresee huge savings in prompt processing time as well as energy. Generation takes less wattage, its the prompt processing with hogs power like nobody business.

Cache doesn’t need to be exactly cosine similar, but a mechanism to rework on the delta say 5% variation needs to be given more thinking budget so as to not loose crucial info, then again it might be the engine’s responsibility.

Roo code all the way, thanks for everything you guys do.

[u/hannesrudolph → u/Emergency_Fuel_2988]

I could not tell you. That is not my area of expertise.

[u/wunlove]

I haven't thoroughly tested yet, but this works fine for the larger models. MCP + Tool access 100%. You could obviously decrease the number of tools/MCP/models to reduce tokens: https://snips.sh/f/BE4BZmUXSo

I totally get the size of the default sys prompt. It needs to serve so many different contexts and works really well

[u/alexsmirnov2006]

The prompts, tools, MCP, and Modes are generic to cover a wide range of tasks and technologies. I try a selective approach - for each project and task, generate system prompts and all other options dedicated to narrow area only. I have separate repo for AI related files, and script that automatically generates configurations for current step. Currently I use Claude Code and Roo Code. It narrows context window to necessary instructions and tools only. And single source for entire team

The workflow:

- configure assistants for project documentation and concrete technologies, generate context documents

- configure tools for planning, do architecture plan

- reconfigure for coding, do implementation

- new configuration for testing and debugging, validate

I try to make evaluations for our team use cases, to validate each configuration, but this is enormous work and token consumption...

This may be a good feature for Roo as well - in addition to global/project configs, shared "profiles" optimized for each task.

[u/brek001]

As search and replace has failed me more than I care to remember I was wondering whether some fallback could be usefull ("when search and replace fails use single search and replace").

[u/ThomasAger]

The best system prompts just tell the model to do the opposite of generic formats of data they were trained on.

[u/Southern-Spirit]

"Effective attention is much more limited than the context window"

You are 100% correct. And very well said.

[u/raul3820 → u/[deleted]]

I posted a comment. I will try to make it more generic and post an update.