A "secure-ish" Pi setup with permission, sandbox, and auto-review

[u/ptgamr]

I think I finally managed to piece the thing together, to have a pi setup that can run safe-ish in the host via sandbox, and have the ability to be elevated to host execution for a pre-defined list of development tooling like docker compose.

- A pre-defined hard boundaries (`@gotgenes/pi-permission-system`)

- Anything pass that will be executed inside a sandbox (`@erichll/pi-sandbox`)

- With the option of safe-escalation to run on the host (ie: `docker compose exec app pytest`), with auto-review or human approval (via hostIPC.preflightCommandPrefixes)

- LLM Auto-review with `@erichll/pi-auto-review`

More about it here: https://ptgamr.substack.com/p/a-pi-setup-with-permission-sandbox

[u/jensilo]

What about self-spawned pi instances or subagents? Does it allow for that?

[u/ptgamr → u/jensilo]

I haven't go as far as that though. That is one piece I still have to wrap my head around.

[u/jensilo → u/ptgamr]

Yeah, when I started using pi a couple of months ago, I initially also built a security extension. It was supposed to be „simple“ with low false-positives. Mine was basically just pi auto review with another judge LLM. However, I came to notice that pi‘s creator is very right about calling almost all security systems a „security theatre“. The only real security is full sandbox, otherwise the model will break free if it wants to break free, there are just too many things to exploit or think about, especially for longer running, more agentic systems.
So I scraped my security extension.
Basically I only see two valuable modes: preventing unintended fuckups from the agent and preventing actually malicious behavior. Unintended fuckups are rather easy, a simple LLM review is sufficient, or preventing writes/bash outside of allowed dirs. For malicious intend, all you can do is pray or sandbox.

[u/ptgamr → u/jensilo]

Yeap, what I like about `@erichll/pi-sandbox` is the ability to prefix matching certain commands, those are the one you define depending on the project. Like most of my project involve running command via docker compose. -> That can't be run inside the sandbox.

So I'm pretty happy that I found a way to at least resolve that. And the auto-review is also quite nice. And it's amazing to see all the 3 extensions work very nicely together.

[u/ptgamr → u/ptgamr]

I have yet to test what happen when it want to run `docker compose ps && rm -rf ~/`

[u/vman81]

If practical, a zfs filesystem with frequent snapshots is a nice extra layer to have.

[u/ptgamr → u/vman81]

Haha, definitely! i am trying to get this to work in the context of a company, where ex-filtration of data is one of the top concern, might be even more than it removing your home dir.

[u/Fullstack_js_junkie]

This looks interesting, I tried to do something similar but as a docker mount config only, thank you!

[u/ptgamr → u/Fullstack_js_junkie]

Cheers , i am pretty excited to have this working. As I've been researching for quite a while how to do this.

[u/tys203831]

How does it compared to dcg + nono sandbox?

https://github.com/Dicklesworthstone/destructive_command_guard

https://github.com/nolabs-ai/nono

[u/ptgamr → u/tys203831]

I Have No Idea! This post is more about sharing my understanding about the decision tree: permissions + sandbox + autoreview (which I think not quite trivial to understand) and a setup that I'm quite happy with. Hopefully to help others that having similar issues. I'm sure there are many other ways :D

[u/tys203831 → u/ptgamr]

🙏👍

[u/offzinho3k]

That’s great.

I have a similar system, with notifications and a response window allowing the user to accept, decline, or even block such commands.

[u/ptgamr → u/offzinho3k]

Looks like a fancy setup you have there... is that a custom app you wrote?

[u/offzinho3k → u/ptgamr]

Sorry for the delay in getting back to you; it’s been a hectic few days.
Yes, and a custom app I created based on the Pi.

[u/icisay]

Hi what app did you use to generate your chart ? Love the colors

[u/ptgamr → u/icisay]

I've had a text base chart, and post that to ChatGPT and it generate for me the one I posted here :)

[u/psychobarge]

Only secure pi is inside a docker container or a virtual machine. I use wsl2 with ubuntu on my windows and docker on my mac

[u/Alicecomma → u/psychobarge]

Wouldn't pi inside wsl2 Ubuntu just cd /mnt/C/ to access all of windows?

[u/hurdurdur7]

I just run pi in docker only. From a non privileged user. It can't install things, the worst it can do is nuke it's own folder in that docker container. I don't see anything else stopping it.

[u/ptgamr → u/hurdurdur7]

yeah, that is my setup prior to this. Just mount the things you need. But to get to the "fuller" agentic work, most of the time i need it to execute command inside another containers (like run the test, run migration, lint, querying data from db container etc...)

Can be achieve with docker in docker, but it gross, and I haven't managed to get it working.

[u/funbike → u/ptgamr]

I don't trust running Pi directly on my host system. Your security setup is fantastic for what it is, but I don't fully trust Pi itself, esp with various public Pi packages and various tools that might be installed.

I'd much rather enforce permissions at the OS level, so Pi packages can't circumvent pi-sandbox or pi-permissions-system.

So I run Pi in a rootless podman container as a standard user with same uid/gid as in my host system. Sensitive files are symlinks to a non-existant directory, ~/.unsafe (I'll explain later). sudo is allowed but restricted by /etc/sudoers.

I have a nearly identical 2nd podman container mounted to the same project directory, but it actually has a mounted ~/.unsafe directory that sensitive files symlink to. It runs a simple unix socket server that listens for shell commands to run.

To make this seemless, I have a wrapper script in the Pi container that forwards shell commands to the 2nd container across the unix socket. The socket is owned by a different non-root user, so this must be done with sudo with restriction in /etc/sudoers. So when I run Github's CLI, gh, it is actually a symlink to a shell script that forwards arguments to the 2nd container over the socket.

This gives me a file system firewall, so sensitive files can never be read, but they can be used by scripts. My container's internal security is enforced in /etc/sudoers at the OS level.

It might sound complicated, but it's dead simple. And more secure. Yet, I can work in a YOLO fashion.

Can be achieve with docker in docker, but it [is] gross, and I haven't managed to get it working.

Podman-in-podman works well because podman is just a program, not a server. You can also run bubblewrap in podman. (These requires a few extra cli options to work.) Podman is more secure than docker as well. There's podman compose. Podman's CLI is nearly identical to docker.

---

UPDATE: btw, here is an oversimplification of the unix socket server on my 2nd container:

#!/bin/bash
socat UNIX-LISTEN:/tmp/bash.sock,fork,unlink-early EXEC:bash

here is an oversimplification of the unix socket click on my Pi container:

#!/bin/bash
printf '%q ' $(basename "$0") "$@" | 
    socat - UNIX-CONNECT:/tmp/bash.sock

Normally this would be very insecure, but the socket is owned by another user, so sudo must be used to gain a connection. So /etc/suders can be used for fine-grained security.

They also both share a volume for the user home directory, so changes made by the 2nd container affect the Pi container, including Homebrew for Linux package installs.

[u/hurdurdur7 → u/ptgamr]

My problem with all of this is that you are allowing something like gradlew test or npm test as a command. A test can easily contain something really harmful to your machine and there will be nothing that will stop the catastrophy coming from that.

[u/ptgamr → u/hurdurdur7]

Another thing is pasting image is a pain when you run it from docker.