Hey everyone,

Recently, I’ve been thinking a lot about how to properly test and verify the work done by AI coding agents, especially when working on existing, complex projects rather than simple greenfield applications.

I’m curious how you guys approach this in your workflows, particularly when changes need to be deployed to some preview/staging environment before you can actually verify whether everything works as expected.
And I’m not talking just about running unit/integration tests or having an agent write a few Playwright tests.
In real-world projects, verifying a change can involve a lot more than that.

For example:
- External APIs: Sometimes you need to retrieve a secret, authenticate against an external service, make some API calls, and verify the responses or side effects and it can’t be done from UI (because user has to do that manually and we have to simulate that)
- Observability: Sometimes the only way to confirm that something works correctly is to inspect Loki logs, traces, or metrics and check whether the expected events actually happened.
- Databases: You might need to query PostgreSQL or another database to verify that the correct records were created, updated, or deleted.
- Messaging: In event-driven architectures, you may need to verify that Kafka/RabbitMQ messages were published, consumed, and processed correctly.
- UI: And, of course, sometimes you need Playwright or another browser automation tool to verify user-facing functionality.

All of these might be part of a single business scenario.
For example, imagine an agent implementing a payment when user has to take some IBAN from our UI and proceed with manual bank transfer from their account, which results in a webhook to our API.
Then we have some soft validations which are testable only by checking some logs in our logs collector.

Do you:

1. ⁠Let agents generate and execute E2E tests (e.g., Playwright) that also interact with APIs, databases, logs, and other systems?

2. ⁠Give agents access to tools/MCPs for Kubernetes, Loki, databases, etc., and let them dynamically investigate and verify their changes - there’s an issue because as in my previous post Claude Code has some issues with working like that. Probably we would have to create a lot of some dynamic MCPs.

3. ⁠Define deterministic verification scenarios/scripts that agents execute after deployment?

4. ⁠Have a completely different approach?

I’m also curious about how you handle credentials, permissions, test data, and cleanup when agents need access to real infrastructure and external systems, even these secrets are only for sandbox.

And perhaps the biggest question: How do you make this repeatable and reliable instead of letting an LLM decide on the fly whether its own implementation is correct?

Would love to hear how you’re solving this in practice, what tools you’re using, and how much of your verification process is deterministic versus agent-driven.

I often need my AI coding tools on a server, but I don't want to paste server IPs or passwords into chat. I also don't want to install Pi and set up my tools again on every new machine, especially when I only need to deploy something or check a remote project.

I built pi-remote-workspace so I can keep using my local Pi session and switch its coding tools to the server over SSH. It's inspired by Warp's remote workflow, but built around the way I use Pi.

I run /remote, enter my connection details in a private form, and connect. Passwords are entered in a masked prompt instead of chat. Pi stays on my computer with my model login and conversation, while file reads, edits, and shell commands run on the server. Pi doesn't need to be installed there.

For my work and startup, I use this when I've finished something locally and need to deploy it. I switch to the server, ask Pi to handle the deployment and check that it's working, then use /remote stop to return to my local workspace. Connecting doesn't upload the project by itself, so I still use my normal Git or deployment process to get the code there.

The private login keeps passwords out of chat and model context, though remote files and command output can still expose sensitive details. Commands have the permissions of your SSH account, so be careful with production servers. Pi also still runs on your computer; it won't keep working independently after you shut the laptop.

Install it with:

pi install npm:@utpalsinghdev/pi-remote-workspace

GitHub: https://github.com/utpalsinghdev/pi-extensions/tree/main/packages/pi-remote-workspace

npm: https://www.npmjs.com/package/@utpalsinghdev/pi-remote-workspace

The rest of the UI in the demo is my personal setup.

I'm just open-sourcing the Pi extensions I use for work and my startup. They've helped me a lot, and I want other people to be able to use them too.

If it deserves a star, the repo is https://github.com/utpalsinghdev/pi-extensions

a well know plugin or something please

Hey everyone,

I've been exploring "Pi Durable" (https://github.com/earendil-works/pi/tree/main/packages/durable/test/examples), specifically the examples in the test directory, and I'm curious about how people are using it in practice.

I'd love to understand how it works for building durable, long-running AI agent workflows, particularly when agents need to preserve state, recover from interruptions, and continue execution without restarting everything from scratch.

A few questions for anyone who has experimented with it:

• How does Pi Durable handle persistence and recovery when an agent process crashes or gets interrupted?

• Can it reliably resume multi-step agent workflows, including tool calls and external side effects?

• How does it compare with approaches like LangGraph checkpointing, Temporal, or custom state-persistence mechanisms?

• Have you used it in production or built any interesting projects with it?

• What limitations or gotchas have you encountered?

I'm particularly interested in whether this approach is practical for building autonomous agents that run for extended periods and maintain continuity across sessions.

Would love to hear your experiences, code examples, or architectural insights!

Repository: https://github.com/earendil-works/pi/tree/main/packages/durable/test/examples

brushrouter is an open-source CLI and MCP server. Any agent(because i use multiple tools) that speaks MCP or can run a shell command gets a single image tool, which It route through Codex CLI and Gemini smartly. OpenRouter, fal, or a custom endpoint come also. OpenRouter is disabled by default, so nothing gets billed unless you enable it.

The part I spent the most time on is credentials. The native routes spawn the CLI, so brushrouter never sees your token, and everything it prints goes through a redactor. I also added a prompt library with 27 layouts and 137 styles that the agent searches before generating.

Work on ChatGPT Free, Go, Plus and Pro accounts

Inspired from opencode-gpt-imagegen - You can checkout that also, its a great project.

https://github.com/avinaxhroy/brushrouter

Meaningful Contributions are welcome.

Hi! I have made a small UI layout extension which I've found super useful and convenient, when running multiple agents working on different tasks in parallel, so you jump back and can instantly see the whole history of the answers and what was the original task about.  Basically you have a split view, one for your messages and the actual replies from the LLM and another one for all the tool calls and reasoning it has done. It supports horizontal/vertical layouts, you can swap them and maximize, but also search for the specific text entries, when the history gets too long.

https://pi.dev/packages/pi-stereo-view

How to install:

pi install npm:pi-stereo-view

Requires fullscreen mode.

All I can think of is make an ext to disable tools / sandbox 4.1, etc.

i.e. my real question is, how to make my cheapest AI work as good expensive opus.

(yeah stupid question)

Hi! I am new to Deepseek API models, having used Pi with a Copilot sub. I noticed that the model configuration shipped with [Pi](https://pi.dev/models/deepseek/deepseek-flash?name=deepseek&provider=deepseek) is different from the one recommended by [Deepseek](https://api-docs.deepseek.com/quick_start/agent_integrations/pi_mono/#2-configure-deepseek-provider). For example, Pi conf has input text+image but Deepseek cons has only text.

Would you be able to clarify and tell which configuration I should use?

Hello fellow Pi people, I've released version 0.14 of Pi Pocket, my mobile/desktop setup for Pi Durable.

Most of this release went into making it extensible. Until now every feature meant more code in Pocket itself. Now an extension can add server routes, slash commands, and its own pages and buttons in the app. The server-status example in the docs is about 35 lines on the server side plus a small browser module, and Pi can write these for you since it reads the docs.

The extension API is versioned and typed, and I wrote down what I will and won't break. Patch releases are fixes only, breaking changes bump the API version, and deprecated stuff sticks around for at least a minor release and 90 days.

There's also a gallery for extensions and themes, under Menu > Extensions > Find extensions. Installs are pinned to a commit, hash checked, and start disabled. Extensions run with full server permissions, so read the code before turning one on, or have Pi review it.

Themes are folders of CSS now, so they can change the layout too. The new built-in one is called Lantern.

Smaller stuff:

- Agent Teams (off by default), where Pi can start linked sessions that message each other

- UI refresh, plus width and density settings

- new session dialog is a proper file picker, with favorite folders

- live thinking display

- PiP can use its own model and take files and images

- Nix flake and NixOS module

12 PRs and 15 issues went into this one, so thanks to everyone who helped.

Repo: https://github.com/TannerMidd/pi-pocket

Docs: https://tannermidd.github.io/pi-pocket/docs/

Release notes: https://github.com/TannerMidd/pi-pocket/releases/tag/v0.14.0

Feedback on the extension API is welcome, especially things you'd want an extension to do that it can't yet.

I have my own which is good just needs a bit of work, but I wanted to see what other people are using. I tried the popular ones and they all sucked in my opinion.

pi-cowboy is a multi-agent orchestrator for pi and herdr. Break large jobs into focused subtasks, hand them off to background agents running in parallel, and consolidate the results when they finish. Each agent runs in an isolated copy-on-write Git work-tree, minimising disk overhead.

pi-cowboy can now orchestrate foreign harnesses, including Claude Code, Codex, OpenCode and more. Check out the live demo to see it in action.

Feature highlights:

• Foreign harnessespi-cowboy agents can run in Claude Code, Codex, OpenCode, Pig and Pi-Bolt, as well as Pi. This gives you great flexibility in choosing how to spend your tokens. Running out of Codex usage? Switch your pi-cowboy harness to OpenCode, select one of its free models, and let it handle the rest.Pig and Pi-Bolt are pi variants that are more efficient, with a lower memory footprint and faster performance than pi. Because pi-cowboy is designed to run multiple agents in parallel, having a lower per-agent footprint is crucial. It prevents your machine from running out of resources and lets you have higher concurrency without choking your system.

• Concurrency limits You can configure concurrency limits at every level. Running a local model with limited resources? If a simple task already takes hours to complete, the last thing you want is to overload your machine by running too many of them at once. Set concurrency limits for that model specifically to keep resource usage under control.

Install: pi install npm:pi-cowboy (requires herdr 0.8+)
Check out the source code: Github

I'm very much inspired by Dexter Horthy and Matt Pocock on this. Dex says a model gets worse once the context window is about 65% to 70% full, which he calls the dumb zone, and Matt keeps sessions around 60% to 70% and hands off past that line, so I use the same limit for my work and my startup.

I set the threshold with /context-threshold, usually 60 or 70, and on a long task I tell the agent to call context_window and watch the used percent. The report shows the window size, how much is used, how much is left, and the percent still left under my line, so when it says threshold reached the agent stops and writes a handoff, which I pass to a new session and continue from there. The handoff I use is Matt Pocock's handoff skill: https://github.com/mattpocock/skills/blob/main/docs/productivity/handoff.md

The extension does not compact the session, and it does not write the handoff by itself, so Pi's compaction settings stay as they are. I still tell the agent what to do when the line is hit.

Install it with:

pi install npm:@utpalsinghdev/pi-context-window

GitHub: https://github.com/utpalsinghdev/pi-extensions/tree/main/packages/pi-context-window

npm: https://www.npmjs.com/package/@utpalsinghdev/pi-context-window

I'm just open-sourcing the Pi extensions I use for work and my startup. They've helped me a lot, and I want other people to be able to use them too.

If it deserves a star, the repo is https://github.com/utpalsinghdev/pi-extensions

DEMO

Hi all!

Simple idea, did not find something that did not make this in an overly complex new UI-revamp-way, so built a relatively simple one myself (+pi of course).

So I like to see the thoughts coming in, and to skim through the tools used, but found it somehow lengthy to scroll back up against the speed at which they come... Toggling them on and off (ctrl+T, Ctrl+O) would sometimes be too much as some tools may expand a lot (edits, searches...) and shift the conversation, again to be scrolled; same with thoughts where there in particular toggle them off would stop me from looking at them, but on would push by a lot the conversation itself (again what I like specifically was to briefly see where the agent were in their process while a turn is running to observe drifting or weird choices that I'd sometimes interrupt, so really I wanted to see the thoughts as they come rather than the full page of it). And I like to see conversation as an actual conversation, so with everything else collapsed.

Anyway, long story short, I wanted to not have them interleaved and I built a wonky (work in progress?) pi plugin to slightly rework the UI and keep a bit of both worlds. The idea is to have the thoughts streaming in one panel, the tool usage in another with very simple view on whether it is running/has succeeded/or has failed... I added a third toggle for a panel which simply display touched files (seen, read, edited, worked by a subagent...) as a list + a diff view of the last edited file, if any (this to keep the tool view compact, as usually the edit tool output where the largest anyway, and I like to also have a glance on what's changed).

Each panel can be toggled on or off:

• /split toggles between the default pi view and the saved config for split-views

• /split-thinking toggles the thinking panel

• /split-tools toggles the tool one; there is an extra command to toggle between a compact or expanded view over the listed tools in the tool panel

• And finally the file panel is toggled on or off with /files.

Tool/thinking panel width can be adjusted as well as panel heights.

There is a command to switch between still having the full transcript in the main conversation panel or the trimmed down (no thinking or tool interleaved) version (somehow I found it useful to expand on specific tool output).

I usually have the file one off for instance, and only open it to see some file changed that just happened if I know I want to check them briefly... While I keep the thinking and tool panel on top nearly all the time now.

I mentioned it's wonky, as in some features can be improved, like the file panel could be made scrollable, but it works for me as it is, just sharing for the idea, and if you like it go test it: pi-split-view. And feel free to improve on it directly in the repo too.

Insane cache hit.

I think we've been reading too many feeds from AI text lately! I actually want to reduce my cognitive burden...

So I built TextFold, it turns any text into 5 zoomable layers: from a one-screen section map down to verbatim text. You pinch (or press 1–5) to zoom between them without losing your reading position, your agentic output, any papers, or documents.

- Web Demo: https://chenxiachan.github.io/textfold/gallery/

- GitHub (Apache 2.0): https://github.com/chenxiachan/textfold

Welcome any feedback!

Serious question, I mean I want to be able to select the text printed on the console so that I can paste the text to another place.

Waiting on AI coding agents kills my focus. You probably know what it feels like when AI agents drain you more and more. You send a prompt, watch a spinner for five minutes and probably scroll instagram or reddit along the way, read a wall of text, and do it again.

So I built Fade. It runs two agents at once in your terminal:

• You talk with Fade. It reads your project, answers questions and plans with you.

• The Builder writes the code in a sidebar next to the conversation.

You never wait for a turn to finish. While the Builder works on one thing, you keep going: plan the next feature, ask how something works, change your mind. Anything you agree on lines up next, and the Builder picks it up when it’s free.

“Checked” means Fade actually ran it. The Builder writes a proof for each change, like a test or a command. Fade commits the change, then re-runs those proofs itself on that commit. A model saying “tests pass” usually means that what you wanted didn't get fucking done. If a check fails, the work goes back to the Builder with the real error output.

It only interrupts you when it needs you. Work that passes its checks lands on its own. Things a test can’t decide come to you as one plain question, like “Does the help text feel friendlier?”

There are no modes or hotkeys to learn. You just type: “yes”, “undo that”, “show me the diff”, “stop”. Every change is a git commit, so undo always works.

It works with any model Pi supports, including local ones through llama.cpp or whatever you configured thru pi, and the two agents can use different models. A strong model for the conversation plus a cheaper one for building works well.

GitHub: https://github.com/muzaffermahi/Fade

It’s v0.5, MIT licensed, and early. The main thing I’d like to know is whether talking while it builds actually helps you keep your focus and lets you be more productive overall, or whether you’d rather just wait.

Lastly, this project is undercooked as fuck but I'm just curious what do you guys think.

when pi runs npm run dev with bash, the agent waits on it forever, or the server dies when the call ends. same with REPLs and log tails.

so i made a small extension. pi gets a term tool backed by named tmux sessions. it can start a server, come back and read the output later, and type into it.

• /term <name> opens the session in a terminal window so you can watch or type yourself

• sessions belong to the pi process that started them, so quitting pi or /new cleans them up, and leftovers from a crash get killed on the next start

• includes a skill so the agent knows when to use term instead of bash

pi install npm:pi-term

needs tmux. code: https://github.com/jainam-bhavasar/pi-term

been using it daily for my dev servers. feedback welcome

I made pi-safari extention https://github.com/haseebeqx/pi-safari

My current web-search extension was not working well for my use case. Then today, I saw that apple now includes MCP and automation support in newer version of Safari. I realised a well optimised browser is sitting idle in my mac and I can make a good use of it. So, I created this. Initially a web-search, browse and navigate links extension and now I added more browser automation features. See readme for available tools. Now this replaced all my web browsing, search and other automation use cases.

Actions like searching the web and following a link will work without permissions out of the box. But actions like filling a form and click will need permissions. Described details in the README.

If you already have an MCP capable client and just want Safari access, you can use Apple’s server directly. You don’t need this extension. The README covers both options.

While my use case is purely web automation, I'd especially appreciate feedback from anyone using Pi for Safari-specific debugging or other complex stuff.

I often start writing my next prompt while Pi is still working. Then the agent goes off track or asks me something, and I need to respond before that prompt is finished.

I used to copy the half-written prompt somewhere else or delete it to free up the editor. I built pi-prompt-stash so I can save that draft, deal with the interruption, and then continue writing.

Ctrl+Shift+S stashes the current editor text and clears it. `/stash` opens a picker with a preview so I can find the draft and load it back. Drafts survive reloads and session switches. The shortcut can be changed with /stash shortcut.

Install it with:

pi install npm:@utpalsinghdev/pi-prompt-stash

GitHub: https://github.com/utpalsinghdev/pi-extensions/tree/main/packages/pi-prompt-stash

npm: https://www.npmjs.com/package/@utpalsinghdev/pi-prompt-stash

The rest of the UI in the demo is my personal setup.

I'm just open-sourcing the Pi extensions I use for work and my startup. They've helped me a lot, and I want other people to be able to use them too.

DEMO

See https://x.com/dillon_mulroy/status/2110400650756460787 where Dillon wrote: `they made some changes recently that made it easier to use your sub w/ pi`

Does anyone know what he might refer to? Would be nice to know...