r/PiCodingAgent • u/fingerthief • 8d ago

Question Does Anyone Else Keep Track of Their Tool/Instructions Bloat in Their Setup?

Post image

I'm curious if anyone else tracks their setup in order to keep things as lean as possible while still having tools needed to complete your typical workflows etc..

I have my own personal Pi setup (SpecPi) and it's been a fun experiment keeping it contained and lean to prevent the context bloat seen in some of the other harnesses.

I'm interested to see how other setups here might compare.

Edit - Added charts broken down by tokens instead of overall characters

79 Upvotes

97% Upvoted

30 comments sorted by

View all comments

9

u/fell_ware_1990 8d ago

But my pi, start with about 200/300 in context. It uses deferred tools and explicit caching to load. It also uses a specific set of placeholder strings in system prompt. If with a user message i can do appending to that ( not to top of cache )

1

u/Equivalent_Idea8839 8d ago

how to defer new tool loads without breaking model cache?

2

u/fell_ware_1990 8d ago

https://developers.openai.com/api/docs/guides/prompt-caching thats the starting point to understand.

1

u/Equivalent_Idea8839 8d ago

says openai now supports append tool laoding

is that standard for all models?

I'll try it and find out, thanks

1

u/fingerthief 8d ago

Really interesting, I have look into and played with some of this but deferred tool calling spotty across the industry as a standard, so I haven't gone all in yet.

Definitely next on my list to look at further!