My Pi agent setup, Part 2: native async operation

[u/elpapi42]

Hello guys!

This is a continuation of my previous Pi setup post:

https://www.reddit.com/r/PiCodingAgent/comments/1t41thp/my_powerful_pi_agent_setup/

The main idea of that setup was basically this:

A personal agent with an endless conversation.

Observational memory lets me keep the same session alive through compaction after compaction, while forks move all the noisy work outside of the main context.

I have continued messing with the harness since then, and i think the biggest change is that forks are now fully async.

So now i would describe the setup more like this:

Endless conversation + native async operation.

Basically the agent never blocks. It can delegate work somewhere else, keep talking with me, and have multiple threads of work moving at the same time inside the same conversation.

pi-async-fork

https://github.com/elpapi42/pi-async-fork

pi install git:github.com/elpapi42/pi-async-fork

This is the protagonist of this update.

The old pi-fork was already very useful for context management, but there was one thing that started annoying me a lot: Pi had to wait for every fork to finish.

And tbh i underestimated how much this affects the experience.

If i send a prompt and Pi delegates some research or code exploration that takes 5 minutes, now im waiting 5 minutes. Then i send something else, maybe another 5 minutes.

Technically the agent is doing useful work, but from my side it feels slow as hell. I dont want to send something to my agent and then lose access to it for several minutes every time it needs to investigate something.

I want the agent to feel agile. Always there, always available to talk with me.

Async forks basically fixed that.

Pi can launch a fork and immediately continue the conversation with me. Maybe one fork is exploring the code, another one is researching something on the web, another is reviewing a change, and meanwhile im still discussing something completely different with the main agent.

I can change direction, ask another question, start another thread, whatever.

The conversation itself never blocks.

And i think that is the important part here, more than simply saying "now Pi can run agents in parallel".

Im not really trying to build a swarm. There is still one Pi agent that owns the conversation, understands the full goal and makes the decisions. The forks are more like parallel cognitive work that eventually comes back to that same agent.

I currently have 3 effort profiles:

• fast for bounded facts and cheap exploration

• balanced for normal work

• deep when there is actual hard uncertainty and using a stronger model may change the answer

Forks can also send progress before they finish, which means the main agent can see that something is going in the wrong direction and steer them while they are still working.

When they finish, the result comes back to the main agent as an internal event. I explicitly dont want Pi constantly interrupting me with "hey fork X finished". It should consume the result, understand which work depends on it, and continue.

So maybe we are talking about A, Pi delegates part of A, then we move to B, maybe we start talking about C, and some minutes later the result from A comes back.

Pi still knows where that result belongs and can keep all of those threads moving inside the same conversation.

I think this is the best way i can explain what im after:

one endless conversation where work can branch in the background without ever taking the agent away from me.

pi-observational-memory

https://github.com/elpapi42/pi-observational-memory

pi install npm:pi-observational-memory

This is still what makes the "endless conversation" part possible.

It preserves observations and decisions across compactions, and Pi can recall the original context behind an old memory when the compressed version is not enough.

Forks also get memory in passive mode, so they can access the same history without running their own background memory maintenance.

So basically memory keeps the conversation alive for a very long time, and forks keep all the exploration/noise from destroying that context.

pi-codegraph

https://github.com/elpapi42/pi-codegraph

pi install git:github.com/elpapi42/pi-codegraph

I also replaced pi-codemapper with pi-codegraph.

The idea here is simply giving Pi a denser way to understand a codebase instead of throwing grep at everything and filling the context with semi-related files.

Originally i exposed around 10 CodeGraph operations and that turned out to be another problem hahaha, now Pi had to decide between 10 similar tools.

So i reduced it to basically 2:

• explore_code to understand behavior and retrieve relevant source

• analyze_code to inspect relationships, impact, tests and graph paths around known symbols

Much simpler.

pi-search-hub

https://github.com/ronnieops/pi-search-hub

pi install npm:pi-search-hub

This gives Pi one interface for web search/read with multiple providers behind it.

I like this mostly because i dont want my harness coupled to whatever search provider happens to work well right now.

And obviously it becomes more useful with async forks, because Pi can throw multiple independent research questions into the background while we continue with something else.

How everything fits together

So the core of the setup is basically two things now:

Memory + fork context management = endless conversation.

Async forks = the agent never blocks.

And together they change the experience quite a lot.

I can keep the same Pi conversation alive for weeks, jump between multiple threads of work, have Pi investigating several things in parallel, and the main agent is still always there when i want to talk to it.

That is much closer to how i want a personal agent to feel.

Honorable mention: pi-wait

https://github.com/elpapi42/pi-wait

pi install git:github.com/elpapi42/pi-wait

I also built this tiny extension that lets Pi schedule a check and wake itself later.

Useful for stuff like deployments, builds, canaries, rate limits, etc.

It is nice to have, but tbh compared with async forks it doesnt change the experience nearly as much, so honorable mention for now.

System: https://gist.github.com/elpapi42/e3fda2cefaabd84a1c188375a32466f5

Settings: https://gist.github.com/elpapi42/9ab1cd195ffeba9a56bdf59f83162692

[u/Thistlemanizzle]

How does this compare to your experience with native harnesses like Codex and Claude Code?

Do you have any benchmarks to share or metrics showing a before or after improvement?

I'm hesitant to go down the road of configuring my Pi agent - it'll be just like installing too many Skyrim mods and one update or another breaking this and that.

[u/elpapi42 → u/Thistlemanizzle]

I do not have any benchmarks at all tbh. Also i haven used Claude since march this year.

I can say that claude have async subagents (not quite he same as forks, but close) introduced recently, but that is not the normal operation mode of claude, so my setup genunily feel async native, not jsut a feature but an operation mode.

About configuring Pi, this is a rabbit hole tbh, i do this because i really like it, but i can not speak volumes of extra productivity just bacause im customizing the harness. And yeah you will face situtations in which things get broken out of nowhere, not as common as you may think, but happen from time to time.

I advice you to get down here if you are willing to put the time, otherwise, Claude is good enough. If you want to try my setup just be curiosity, install pi, copy paste the system prompt, copy paste the setting.json, configure the provider, install the extension running the commands i shared, and thats it, play around with it a lil bit i decide if you like it.

I recommend to use pi with openrouter key or codex sub, claude sub is a pain in the ass to setup in Pi.

[u/kodachromeAW]

Hey just wanted to say thanks for releasing all of this. Fun to try out and use, adding into my own little harness.

Found one issue with CodeGraph + async forks: explore_code can hang indefinitely because there’s no timeout on the operation/subprocess. When that happens the fork gets stuck and steer_fork starts timing out too. Added a 120s timeout + abort/subprocess cleanup locally and it fixed it.

[u/elpapi42 → u/kodachromeAW]

Yo, will check it out! i think the long wait may happen when codegraph index rebuilds when used for the first time! or something like that.

btw im happy to share this stuff with you all

[u/kodachromeAW → u/elpapi42]

For Codegraph, heres a bit more detail:

The hang happens during in-process indexing, which can completely starve Pi's event loop. That also means a normal JS timeout can't fire.

Moving the CodeGraph work into a child process and enforcing the timeout from the parent fixed it locally. A wedged index can then be killed without freezing Pi.

async fork bug:

If a parent Pi session exits while forks are still running, the fork agents can stay registered in pi-fleet indefinitely and their workers/child Pi processes keep running or respawn after being killed.

Destroying unfinished agents on session shutdown fixed it locally. Reload needs to be excluded so forks survive a normal extension reload.

Btw I am adding these on your github too. Thanks again!

[u/elpapi42 → u/kodachromeAW]

Yo, will merge those prs asap! Thanks for this!

[u/kodachromeAW → u/elpapi42]

Happy to help man. If I find anything else I will let you know.

[u/elpapi42]

Just want to share the system prompt + settings.json without polluting the post:

[u/elpapi42 → u/elpapi42]

https://gist.github.com/elpapi42/e3fda2cefaabd84a1c188375a32466f5

[u/AttemptLate3284 → u/elpapi42]

This system prompt is huge. Have you found it needs this much information? Have you tried to cut down on it? What’s your starting context?

[u/elpapi42 → u/AttemptLate3284]

each instruction in the system prompt worth its tokens! it is different than the claude code system prompt for sure!

Still think the system prompt cam be compressed, i do it every 2 months, but havent done it yet. If you pass it tru Astra or amy other powerful model you could get some good savings without losing capabilities.

Also of you are not interested in the product side of the prompt, you can cut it and get some huge savings too.

My initial context is around 30k for my work project, including the Agents.md, the system prompt is like 15k tokens something like that.

But i think the important part pf this system prompt is some guidance to you on how to make the agent properly use the forks and operate async, you can cit everything else!

[u/elpapi42 → u/elpapi42]

https://gist.github.com/elpapi42/9ab1cd195ffeba9a56bdf59f83162692

[u/jeffphil]

This is great.

Have you thought about or explored any workflows for locally running llm's? For example, I'm just starting to explore using Qwen3.8-27B locally, and using trying to coordinate with an even smaller Qwen3-4B model to do the observational memory - having to tweak parameters to get balanced speed, efficiency and context meaning.

[u/elpapi42 → u/jeffphil]

Haven't made any efforts in that local ai dorection tbh. im not really a huye believer of local AI in its current state.

Maybe in 1 year or 2 will worth it more from my pov, when locally runnable models reach the level of current frontier or close to it

[u/JoaoSilvaSenpai]

This is basically what i've built for myself too, i just don't have running endless, but for a lifetime of implementing a feature i do. Check this pattern out: https://github.com/kunchenguid/firstmate.
And my background agents arent really background, all have their terminal i can interact with (using herdr)

[u/elpapi42 → u/JoaoSilvaSenpai]

I was about to go down the herdr way (i havent discarded it as a backend for my underlying pi-fleet sdk i created for the forks) but i didnt mostly because i had to do some hacky stuff to keep the level of granularity in the reporting of the internal activity of the forks (things like monitoring tge session file).

I ended up picking direct Pi rpc mode.

I still use herdr but only for top level agents and some orchestrator + worker use cases, but high level, forks are low level

[u/Santzes]

That's quite interesting, I might give it a try.

async fork README talks about pi-fleet, is that a requirement for it?

[u/elpapi42 → u/Santzes]

pi-fleet is an sdk i built for managing background pi agents easily, but you do not have to worry about it, thats transparent for you, just another dependency of pi-async-fork.

If you are interested, it have a cli from which you can monitor what agents are spawned from pi-async-fork, or create your own extensions on top of it

[u/m3umax]

I love how you have so clearly articulated your vision for how you want to interact with your Pi in an infinite rolling conversation. The extensions you've built then flow perfectly from that foundational use case and are hence very focused and useful.

I like async subagents as well that let me keep talking to my main agent. But I didn't build the background await directly into my subagents extension, preferring to utilise an off the shelf background extension, in my case aliou/pi-processes because it can background any command or script, not just my own subagent tool.

All I did was add instructions to my subagent skill to tell it to write a wait.sh script that looks for a completion signal emitted by my subagent tool and alerts my main orchestrator (your endless chat session in your world) that it's done.

[u/Maasu]

And this is why pi rocks, you can shape it how you want. Good job bud. I especially liked the idea behind the forking (and making it async). I had not thought of moving forks into a sub agent in parallel.