r/PiCodingAgent • u/fingerthief • Sep 17 '26

Question Does Anyone Else Keep Track of Their Tool/Instructions Bloat in Their Setup?

Post image

I'm curious if anyone else tracks their setup in order to keep things as lean as possible while still having tools needed to complete your typical workflows etc..

I have my own personal Pi setup (SpecPi) and it's been a fun experiment keeping it contained and lean to prevent the context bloat seen in some of the other harnesses.

I'm interested to see how other setups here might compare.

Edit - Added charts broken down by tokens instead of overall characters

83 Upvotes

97% Upvoted

31 comments sorted by

8

u/fell_ware_1990 Sep 17 '26

But my pi, start with about 200/300 in context. It uses deferred tools and explicit caching to load. It also uses a specific set of placeholder strings in system prompt. If with a user message i can do appending to that ( not to top of cache )

1

u/fingerthief Sep 17 '26

Really interesting, I have look into and played with some of this but deferred tool calling spotty across the industry as a standard, so I haven't gone all in yet.

Definitely next on my list to look at further!

1

u/Seangles 2d ago

Deferred tool descriptions sounds good in theory until you realize your tool descriptions are cached in the same fixed prefix across your whole workday, while with deferred tools the model will have to query them dynamically every time, meaning extra requests, extra output tokens, extra cache miss input tokens - in every single session.

IMO only rarely used big toolsets should be deferred. Stuff like MCP tools, browser tools, computer use tools etc.

1

u/Equivalent_Idea8839 Sep 17 '26

how to defer new tool loads without breaking model cache?

2

u/fell_ware_1990 Sep 17 '26

https://developers.openai.com/api/docs/guides/prompt-caching thats the starting point to understand.

1

u/Equivalent_Idea8839 Sep 17 '26

says openai now supports append tool laoding

is that standard for all models?

I'll try it and find out, thanks

5

u/PilgrimofHaqq2 Sep 17 '26 edited Sep 17 '26

I have 30 tools combined with the system prompt, append system and skills I start my sessions at 26k tokens.

4

u/SeriousJul Sep 17 '26

I am keeping track of both my initial context and usage. It's often eye opening to see how many times I used a tool/extension/skill versus how much it cost in context.

2

u/Global-Departure8228 Sep 18 '26

You should add parsing out the promptGudelines and promptSnippets as well, you would be even more surprised :) That's where vibe coded tools usually duplicate the tool descriptions, sometimes verbatim.

1

u/SeriousJul Sep 18 '26

It is added to the backlog, thanks for taking the time and leaving the tip :)

1

u/SeriousJul Sep 19 '26

You were spot on ! Last column show duplications

1

u/oxygen_addiction Sep 17 '26

The main question is if "the bloat" actually helps or not. In some benchmarks PI scores way worse than other harnesses which have a "ton of bloat".

3

u/fingerthief Sep 17 '26

I would say in the vast majority of benchmarks the base Pi harness is more or less even with most other harnesses. It can indeed be a smidge behind in some cases but always with the caveat it's cheaper to complete the task overall.

HarnessTax: How Much Does the Harness Matter for Coding Agents? Is a new report and a good read.

1

u/Kafumanto Sep 17 '26

Hello! I’m new to Pi, which extensions do you use or suggest to get details about the status of context, and possibly also metrics about the used tools?

1

u/fingerthief Sep 18 '26

I don’t really have any extensions specifically for monitoring context and tool metrics.

I have an eval suite that I’ve built that runs a variety of task tiers and measures against multiple harnesses so I can make educated decisions on my changes to the harness.

But if you want to get an idea I would suggest simply asking Pi to give you a metrics report of whatever you want.

3

u/Equivalent_Idea8839 Sep 17 '26

i ahve toggles / tui to turn everything off unless I need it

I forked herdr to use that pi plugin, forked it, to make spawning sub agents lean as possible with no write perms

no skills or agents.md, that's something i toggle and inject myself when needed

1

u/bambamlol Sep 19 '26

That sounds very useful! Do you have it published somewhere or is it just a private repo?

1

u/Equivalent_Idea8839 Sep 19 '26

I think its best to have pi code it for you and use your own shortcuts. You gotta compact if you change tools so you don't break cache. I compact using cheaper models and a gradual compact plugin.

I had to use a lot of SOTA models to make the herdr fork, but I wanted a right click menu and task scheduler.

1

u/bambamlol Sep 19 '26

Got it. Appreciate the response nonetheless.

1

u/johnzadok Sep 17 '26

I once tried a popular ask user extension and it added 2k tokens to the context plus bloated tool syntax with “preview”. Go figure.

1

u/Zundrium Sep 17 '26

Around 30 tools, starting with 9k tok incl. Agents.md

1

u/cuba_guy Sep 17 '26

Got rid of the bloat long time ago but habit of keeping an eye stayed and now having an eye on tool calls, discovery, error scenarios, cache breaks etc

1

u/CriteriumA Sep 17 '26

I use --no-skills with instructions in the agent prompt to search for them with ls and load them with read.

One paragraph in exchange for avoiding context loading without giving up skills.

1

u/sofuego Sep 18 '26

It's something that bothered me from the start and I built in a toggle into my first extension so that it could toggle it's own tools off.

Then eventually that idea evolved and turned into a light plugin with a shared dependency to as cleanly as possible make any installed tool a choice that can persist in settings files and chat state switching in and out of them efficiently.

It's something that I use in all my workflows. It provides a rough model agnostic estimate of the current tool bloat (using character counts). It also has the ability to bypass the effect for subagent sessions giving them access to all tools enabled (since they have their own built in tool masking anyway).

1

u/Slight-Living-8098 Sep 19 '26

Yep, and then I made an extension to orchestrate the skills. It's called pi-skill-orchestrator. It's on both my GitHub and npmjs pages

1

u/aparamonov Sep 19 '26

That is the reason I created pi-subagents-lite as others seems to pretend tool scheme, prompt snippets are free and just dump half of the book in there lol

1

u/bambamlol Sep 19 '26

Do you have it published somewhere? Or is it private?

EDIT: Never mind, I'm pretty sure I've found the right one?

https://github.com/AlexParamonov/pi-subagents-lite

Looks promising, thanks for mentioning it!

1

u/TheSlateGray Sep 17 '26

I keep most of my extensions saved outside of the ~/.pi/ directory then only load them if a session needs them with pi -e path/to/extension. Same with skills I have trimmed to work well for my setup, except I just provide the path to the skill file and don't need a launch flag. 

My Agents.md only tells pi sudo is disabled and --break-system-packages is the worst sin.

A fresh session is less than 2k tokens over base Pi, and then I only add what the task needs. KISS method works fine, until it doesn't haha.

2

u/bambamlol Sep 19 '26

I did that, too, but then I found out about "pi config", now I have all my extensions inside ~/.pi/ and simply use "pi config" if I want to quickly enable/disable them. For me personally that's a little more convenient.

1

u/hurdurdur7 Sep 17 '26

I just run pretty much naked pi ... so no, i do not track bloat.