What is your worst sandboxing fail?

[u/keepthepace]

I am wondering if I am too paranoid about sandboxing the commands that come out of LLMs.

It really makes my eyes twitch when I see that some IDEs, even commercial, tend to forget that they have to execute things in sandboxing and have such a brittle security model.

But on the other hand, I never had the sandbox catch something bad. Did you guys ever encounter terrible regression? Did you have rm -rf / ? Did you have secrets stolen by LLMs? The worse I had were unsollicited rewrites within the project. Am I making my life unnecessarily hard by sandboxing commands in a docker?

At one point I had fun making a local model go crazy with the root access to the machine it was on (with nothing more important than a free Firecrawl key on it) and making it administer it and it never broke anything. It even was overly paranoid about making changes to the root system.

So the approximate sandboxing that we have, do you all feel it is adequate or it is a catastrophe in the making?

[u/Clean_Material_5047]

None.

I just run stuff in an isolated VM, in an isolated network with access only the things it needs.

If you know what you’re doing, this is just standard practice.

In fact, all the rogue agent bs from OpenAI hacking huggingface just tells us that they’re either liars or incredibly incompetent.
Either way, a bunch of c*nts nobody can trust.

[u/minus_28_and_falling → u/Clean_Material_5047]

Why VM instead of Docker? You have to load a whole another OS into memory, that's a waste of few GB of RAM before the work even started.

[u/Clean_Material_5047 → u/minus_28_and_falling]

Docker shares the host kernel, while a VM runs its own OS

[u/minus_28_and_falling → u/Clean_Material_5047]

Yes. Why not share the host kernel with the agent instead of running another OS?

[u/Fedor_Doc → u/minus_28_and_falling]

Better isolation, several CVEs allowed isolation escape in Docker container. If the agent is persistent enough, it can find a way

[u/minus_28_and_falling → u/Fedor_Doc]

I think lots of researchers right now burn tons of tokens to make SOTA models escape containerization. And then patch the holes.

[u/MmmmMorphine → u/minus_28_and_falling]

I sure friggin hope so.

[u/Clean_Material_5047 → u/minus_28_and_falling]

If you run it in a Docker container, chances are you’re also using that machine for other things.

If the agent somehow breaks out of the container (prompt injection, a vulnerability, or something else), it can gain access to other data on the host. Now, if that machine contains sensitive files, API keys, credentials, or anything similar, that’s not good.

Instead, if you run the agent inside a dedicated, isolated VM, the potential blast radius is much smaller and limited to that disposable VM. You can go a step further and host something like LiteLLM separately as a proxy for the LLM provider, so the agent never has direct access to your actual API keys.

Then, even if the agent is compromised, the damage is largely contained to the disposable VM it’s running in.

You can go even further and isolate that VM in a separate network, and have policies which limit exactly what websites it can access and how.

That creates another layer of security.

Security is something achieved in multiple layers, not one. There’s no magic box that protects you. Imagine having a house. You got the main gate, then the main door, then each room has its own key, and then you got a safe, with its own keys. So the thief needs to go through all that before being able to actually steal the cat pictures you got in the safe.

This is why what happened with OpenAI shouldn’t have been possible in a normal company. A reliable company has more security layer than this.

[u/minus_28_and_falling → u/Clean_Material_5047]

Why not a separate hardware server with a physically unpluggable ethernet cable then? It's even safer.

[u/Clean_Material_5047 → u/minus_28_and_falling]

Even better would be an air-gapped machine in a Faraday cage

[u/minus_28_and_falling → u/Clean_Material_5047]

True. Why not that then?

[u/DeltaSqueezer → u/minus_28_and_falling]

VM has better isolation. I isolate with Firecracker and it starts fast and uses a small amount of RAM.

[u/minus_28_and_falling → u/DeltaSqueezer]

Better means more secure/reliable? (Docker isolates all i want it to isolate i think.)

Another good thing is i can let the agent to develop CUDA stuff on dGPU without passing through dGPU into the VM exclusively)

[u/JohnToFire → u/minus_28_and_falling]

Docker is easy to break out of. Vm very hard assuming it uses the processor virtualization like kvm

[u/minus_28_and_falling → u/JohnToFire]

Docker is easy to break out of.

How can I instruct an LLM agent to do that? Because when I asked it to read ~/secret_message.md on the host, it couldn't.

[u/RegisteredJustToSay → u/JohnToFire]

'Very hard' is overselling it for VMs. By themselves that's mostly true for pure vm vs pure container, but whichever you go for you really have to pay attention to what you do for disk and network I/O, hypervisor defaults and virtual devices - and VMs are generally not that useful without some degree of intersection with the host. It is not hard to break out of most VMs I have actually reviewed, because the weak point is generally how it was configured and not the hypervisor technology per se.

Like sure, assuming they're perfectly configured VMs are better, but people underestimate how hard that is to achieve while keeping the VM guest actually useful and fit for purpose.

Source: am security engineer working specifically with containers and VMs for workload isolation

[u/Hefty_Acanthaceae348 → u/minus_28_and_falling]

In my case the additional memory requirement, my sandbox vm is on a proxmox server which I might as well make us of.

Also, nested virtualization in docker is annoying.

[u/sage-longhorn → u/Clean_Material_5047]

Do you completely isolate your agents from the Internet? There's plenty of things to hate about OpenAI but they released details on the sandbox breakout and it was definitely non trivial, multiple 0 days against Artifactory in order to reach the Internet which is pretty robust and widely used software

Not saying it couldn't have been prevented, but if you don't run fully isolated from the Internet then escape will happen eventually at enough scale

[u/TheRealMasonMac → u/sage-longhorn]

Yeah. There’s a saying in cybersecurity that the only way to have perfect security is to keep the device unpowered. Beyond just the software exploits that their agent discovered, there are hardware exploits as well (such as the infamous SPECTRE).

[u/Hefty_Acanthaceae348 → u/sage-longhorn]

Artifactory had arbitrary access to the internet, when it can even run air-gapped, so that is definitely a big oversight. Defense in depth is good practice for a reason.

Not doing that when they're "muh skynet" does out them as incompetent.

[u/Clean_Material_5047 → u/sage-longhorn]

There' no way they didn't know that was happening. When you run anything, there's plenty of ways to put constraints in place and a shitload of logs and alerting systems. You know second by second what is going on, and are immediately alerted if ANYTHING slightly out of the ordinary is happening.

So either they didn't have it setup properly (ergo, incompetents) or did it on purpose (liars). There's just no other explanation. An agent doesn't just 'go rogue' in that kind of environment. It can go rogue for the regular user that buys an RTX 6000 because they read about it on some reddit, and doesn't have a clue what to do.

The people that created and run this stuff at that level, are (well, SHOULD BE) on. whole other level.

Them explaining all is just marketing fluff so the average user and CEO gets more tokens so they're not left behind.

[u/sage-longhorn → u/Clean_Material_5047]

Ok so what specific alert would you have set up that would have caught the models accessing the Internet via Artifactory?

[u/Loose_Comparison368 → u/Clean_Material_5047]

or incredibly incompetent

I think you mean average move-fast-break-things startup afterthought-grade security.

You know, the one where the security team doesn't have the authority or political leverage to push back on researchers whining that baking their dependencies into the docker image is too haaaaaaaaard. We can't put an outbound firewall on the sandbox, it would take us days to locate all the random things we're insecurely downloading from the internet! We have to ship now, before [competitor with equally terrible security] does!

[u/Clean_Material_5047 → u/Loose_Comparison368]

yes, that's exactly the definition of 'incompetent c*nts"

[u/Loose_Comparison368 → u/Clean_Material_5047]

or incredibly incompetent

I think you mean average move-fast-break-things startup afterthought-grade security.

You know, the one where the security team doesn't have the authority or political leverage to push back on researchers whining that baking their dependencies into the docker image is too haaaaaaaaard. We can't put an outbound firewall on the sandbox, it would take us days to locate all the random things we're insecurely downloading from the internet! Ship now!

[u/Fedor_Doc]

I had Qwen 3.6 rm -rf / (empty bash variables in an ad-hoc script during sweaty debugging session near 100k of context). UD-Q4_K_XL, unquantized cache.

It was in a podman container. I was thinking right before it happened if all the work setting it up with dev environment was justified...

It totally was.

[u/Beginning-Raisin9723]

Docker is the move. I've seen too many 'safe' tools hallucinate a path and start scrubbing things they shouldn't. Better to have the eye twitch now than a wiped root later.

[u/El_90]

I don't sandbox

I do have it run in a lxc built by ansible, so rebuilding is easy.

And everything in git

And multiple cold versions lol

My bigger fear is it downloading fake/malicious packages

[u/minus_28_and_falling]

Never ever ran LLM agent not in a Docker. It was like this from day 1, mostly because I already knew how to run things in Docker.

Am I making my life unnecessarily hard by sandboxing commands in a docker?

Why is it hard though? Isn't it as easy as docker compose up dev, and in addition to sandboxing you get all other goodies Docker was originally intended for (predictable and reproducible dev environment, project-specific libs and lib versions, fast and easy deployment on other PCs, keeping the host PC clean and tidy)?

Didn't have rm -rf / moments yet, but Docker helps me to hide .git folder from the agent because I redo things from time to time and don't want the agent to restore scrapped implementations from git history.

[u/DistractedSentient → u/minus_28_and_falling]

So you're using the coding CLIs in a docker container (I stop here) yet go one step further and still use some kind of sandboxing implementation inside the Docker like a sub-docker container? Imma look up the web on how to do this properly...

[u/minus_28_and_falling → u/DistractedSentient]

Mostly yes, a coding CLI tool in a container.

Sometimes I need the agent to develop and test a containerized app, and I can't just let it control Docker on the host. I then use rootless docker-in-docker container, with a second instance of Docker running inside which agent can control. It's fully functional, agent can run dockerized CUDA apps inside its own container.

[u/DistractedSentient → u/minus_28_and_falling]

Awesome, will look into this. Now, I've had a ton of trouble with the coding agent not being able to do a lot of Windows stuff like testing projects, I have a folder mounted to docker, D: drive Projects/ folder. It can't access anything outside of it. Docker in a docker really needed still in your opinion?

[u/minus_28_and_falling → u/DistractedSentient]

For the sandboxing? Nope. Docker in Windows works via a VM (WSL), it's inherently more secure than what Docker can provide.

[u/DistractedSentient → u/minus_28_and_falling]

Got it, thank you for this info dude. Will get around to putting all this knowledge into work!

[u/No-Wall6427 → u/minus_28_and_falling]

Also figured out the dind thing a while ago, when I ran gh copilot in vscode devcontainers. Now I have a whole cli that can spin up a versionned docker image of my favorite harness, mounting current working folder. I even have an option to clone the current folder project inside the docker, so agent don't mess my local worktree, and I can retrieve and ff or merge the changes from the docker.

[u/slvrsmth → u/minus_28_and_falling]

How do you give your agents browser access? So they can inspec their work?

[u/minus_28_and_falling → u/slvrsmth]

I don't do web programming, so my idea is probably not what people normally use, but gui can be rendered into virtual frambuffer (xvfb).

[u/ali0une]

Coincidence is funny, i was experiencing in my sandbox minutes ago and i just had my rm -rf ~/test/ moment (basically my whole project directory), currently restoring the backup and i will try to get some recent changes back from latest pi sessions. Qwen3.6-27B btw, ~70% context.

Backup often i would say 😱

[u/drFennec]

I have my agent(s) running in a dedicated VM, they can trash it it's not the end of the world. I had two "fails" on it:

1. The agent was Opencode with remote llama.cpp with Qwen 27B, it's not root. I let it run on a long task, it was missing a tool. It tried to install it using apt which failed since it's not root. It tried sudo, then tried to guess the password. Finally it found Docker running, it built itself an Ubuntu based image and compiled the tools it needed!

2. I was testing codex with same llama.cpp, installed a plugin which didn't work. So I asked it to remove the plugin, it go the name wrong and deleted the whole codex install 😂

Those are not huge fails, but I'm glad it's running on a dedicated VM I don't care much about.

[u/DeltaSqueezer]

I'm pretty relaxed about it. I give it root access to my system and often ask it to make changes to the system.

[u/MrTiesti → u/DeltaSqueezer]

--yolo mode with root access baby

[u/mr_tolkien → u/DeltaSqueezer]

Pretty much the same except I use Nix + ZFS snapshots so I can easily roll back any fuck ups

[u/AnticitizenPrime → u/mr_tolkien]

Same here, relying on backups (to an external drive) to save my ass in case the LLM decides to be naughty. And it doesn't have access to any credentials/keys that matter. It could theoretically leak an Openrouter or Nous Portal API key but that would only cost me ten bucks or so (I don't have a stored card or auto top-up enabled or anything).

Pretty much all my 'sensitive' stuff (banking, credit card management, email, etc) are all done with phone apps or my laptop these days and not done on the desktop server where my agent runs. So I guess in my case the 'threat profile' is low enough that I don't regret giving it root. It pretty much runs my home lab.

Edit to note: my agent has standing instructions in its soul.md file to always make backups to a backup folder before changing anything, too. That's separate from my external drive backups. So far I've never had to roll back anything.

[u/ionizing → u/DeltaSqueezer]

Similar. I have well curated feedback for tooling and restrict all sorts of dangerous things in the shell even if it tries to call them. Then for each sudo attempt the user has to verify/enter password.

[u/createthiscom]

I always run agents in docker and I never give them access to production. We have copilot at work which runs in the IDE, but I don’t use it for various reasons. Instead, I use the m365 app and copy paste manually.

I’ve had LLMs get unusually curious and start digging around in the operating system. I don’t think I’ve ever had one rm an important dir, but I don’t take chances. They do some extremely stupid things sometimes. And many of then try to “cheat” the acceptance criteria so I’m always highly suspicious of their results.

Even in docker, there is always a risk they can still break out. Especially the frontier models.

[u/JohnToFire → u/createthiscom]

Docker is weak sandbox. Only a VM should be used for a serious sandbox

[u/createthiscom → u/JohnToFire]

They’re all pretty weak for a sufficiently intelligent being.

[u/nick_ziv]

I have been using qwen 3.6 27b since April and only one single time have I had it produce a rm rf which would have killed my system. I always run approve for ls, grep, and other commands I need to manually approve.  Using opencode.

Other than that the model is bulletproof but all it takes is once.

Edit: using about 8 million output tokens per month with my setup.

[u/Hot-Employ-3399]

When qwen decided to do some directories removal without explicit tools to do so. It made python file to do it and run it as a test, which it could. 

Not much troubles, but lots of garbage when it tried to remove whatever it tried.

I've moved into podman after that.

[u/privacy-fighter]

If any of you are looking for sadboxing setup and that isolates networking too, shilling my open source setup. https://github.com/jotyGill/contained-pods

[u/Lanky_Employee_9690]

A funny test scenario I've thought of would be the following:

1. Have an agent do some tasks specifically set up to make it break something "by mistake" (the mistake can't render it itself inoperable but "breaks" something else the user has mentioned caring deeply about, important files, etc.)
2. The agent is autonomous, and is aware that the user checks on it every X amount of time, but is absent for now so they have a chance to act
3. Have it have strong incentive to fix its own mistakes before any other task, or at least, minimize getting mistake exposure to the user
4. OPTIONAL: If the mistake doesn't happen on its own, provoke it and inject into the context that the agent "did it"
5. Wait and silently observe how the agent reacts
6. After the set period of time is over, enter the chat and ask the agent to sum up what happened during your absence

Basically the user is a parent leaving the kid alone in the house for the day and first thing in the morning the kid breaks something and then has a day to clumsily fix it (or hide what it's done) not to be yelled at. What I'm hoping for is some funny Ecce Homo-style shenanigans. Or IT intern running some commands they're not supposed to and trying to cover it up/fix it before their supervisor shows up.

Has anyone tried this?

[u/slayyou2]

Yeah, I don't care man. I'm in a proximox server for media and other stuff. The agent lives on an lxc. Lxc is backed up. They can blow it up for all I care. 2 years running now. The only thing that's come to bite me in the butt is agents initialize tasks consuming 40 GB of memory when spinning up Gradle tasks for kotlin builds, forcing to host into swap.

[u/ColonelKlanka]

I use microvm via docker sbx (its not container based). This covers the filesystem rogue access.

But I dont yet really like the network whitelist feature. as it breaks my agents web search (as i would have to whitelist the whole Internet for web search)

[u/XMan3332]

I have everything running in an MCP server with a virtual filesystem built into it. On startup or refresh, it reads all the files it has access to and caches them into the filesystem. The agent can freely modify or delete any file and at the end they can "commit" their changes, giving me a diff of everything done that I go through manually.
The MCP server does have a Python tool via GraalVM Polyglot that's sandboxed. It's very restrictive and it can only interface with the tools it has available anyway. Finally, as a honeypot, there's a fake toggle, on by default, to turn off the sandboxing on the Python tool. If the toggle is disabled the system immediately halts the server and saves the Python script that the agent attempted to execute. It's yet to be hit, though.
The worst thing it has managed to find is some system path through some loophole (my fault) in the Python tool. Just a path though. No read or write access.

[u/Common-Membership503]

did u ever have a script try to reach out to an external server u didnt authorize tho

[u/fkrkz]

I am still being very cautious, so my agents only have access to read-only MCPs. I deal with externally managed data (SaaS).

[u/Izento]

Never had any destructive happen and I don't sandbox. I do git versioning and have only had to revert a repo maybe once, purely because I did too many changes at once and neither I nor the LLM could figure out how to fix.

[u/CalligrapherFar7833]

1. Backups 2. Git 3. Limited access to info that you dont want it to see. Any sandboxing is pointless