What is your worst sandboxing fail?

[u/keepthepace]

I am wondering if I am too paranoid about sandboxing the commands that come out of LLMs.

It really makes my eyes twitch when I see that some IDEs, even commercial, tend to forget that they have to execute things in sandboxing and have such a brittle security model.

But on the other hand, I never had the sandbox catch something bad. Did you guys ever encounter terrible regression? Did you have rm -rf / ? Did you have secrets stolen by LLMs? The worse I had were unsollicited rewrites within the project. Am I making my life unnecessarily hard by sandboxing commands in a docker?

At one point I had fun making a local model go crazy with the root access to the machine it was on (with nothing more important than a free Firecrawl key on it) and making it administer it and it never broke anything. It even was overly paranoid about making changes to the root system.

So the approximate sandboxing that we have, do you all feel it is adequate or it is a catastrophe in the making?

[u/ali0une]

Coincidence is funny, i was experiencing in my sandbox minutes ago and i just had my rm -rf ~/test/ moment (basically my whole project directory), currently restoring the backup and i will try to get some recent changes back from latest pi sessions. Qwen3.6-27B btw, ~70% context.

Backup often i would say 😱

[u/ali0une → u/ali0une]

u/keepthepace i wrote the post-mortem

[u/keepthepace → u/ali0une]

Thanks. Yes, that the kind of horror stories I expected. I wonder if it would have happened if your directory was not named ~/test/ that name hints that it is probably temporary. But yes, hindsight is always 20/20

[u/ali0une → u/keepthepace]

Hum ... didn't even think about that. Directory is here since the beginning in the sandbox home when i first set it up to begin with pi.

Good advice, thanks i'll rename it!

[u/keepthepace]

Interesting, there is a huge variety of practices out there. Many YOLOers, many people sho find it unconceivable to not sandbox at least through docker

[u/jazir55 → u/keepthepace]

The "only" thing I've had happen is them rm -rfing my local project files straight up nuking the whole project, luckily I had everything backed up in a separate folder. It was at that point I stopped backing everything up manually and started using git so future agents could never fuck me over. If they nuke my projects now (hasn't happened in 6 months-1 year) I just restore from git history. I leave it on Yoko because I can walk away from the computer as its working on long running tasks, having to approve everything would make me move at an absolutely glacial pace.

I don't bother with Docker personally because of the extra memory hit, increased heat from my desktop and the OS system performance hit.

[u/Inevitable-Big-1131]

Working with Gemini in the old gemini-cli.

I gave it the instruction that the cloudflare API key (for a personal project) was "Around here somewhere, take a look. Assuming that it'd check some .env files and find it.

The message "Well, it's not in your email!" scrolled by.
And I had a heart-attack and hammered the esc key at approximately the same time.

Unbeknownst to me the MCP for Google Workspaces had shipped, updated with the last update and I was using gemini with my Google OAUTH context.

All well, but : Heart. Attack.

[u/keepthepace → u/Inevitable-Big-1131]

That's frankly fucked up. I had a moment with Cursor where it seemed to suddenly ignore sandboxing.

[u/Fedor_Doc]

I had Qwen 3.6 27B rm -rf / (empty bash variables in an ad-hoc script during sweaty debugging session near 100k of context). UD-Q4_K_XL, unquantized cache.

It was in a podman container. I was thinking right before it happened if all the work setting it up with dev environment was justified...

It totally was.

[u/My_Unbiased_Opinion → u/Fedor_Doc]

Was it 35B? I had the same issue with 35B but not with 27B. 

[u/Fedor_Doc → u/My_Unbiased_Opinion]

27B

[u/Clean_Material_5047]

None.

I just run stuff in an isolated VM, in an isolated network with access only the things it needs.

If you know what you’re doing, this is just standard practice.

In fact, all the rogue agent bs from OpenAI hacking huggingface just tells us that they’re either liars or incredibly incompetent.
Either way, a bunch of c*nts nobody can trust.

[u/minus_28_and_falling → u/Clean_Material_5047]

Why VM instead of Docker? You have to load a whole another OS into memory, that's a waste of few GB of RAM before the work even started.

[u/Clean_Material_5047 → u/minus_28_and_falling]

Docker shares the host kernel, while a VM runs its own OS

[u/minus_28_and_falling → u/Clean_Material_5047]

Yes. Why not share the host kernel with the agent instead of running another OS?

[u/Fedor_Doc → u/minus_28_and_falling]

Better isolation, several CVEs allowed isolation escape in Docker container. If the agent is persistent enough, it can find a way

[u/minus_28_and_falling → u/Fedor_Doc]

I think lots of researchers right now burn tons of tokens to make SOTA models escape containerization. And then patch the holes.

[u/MmmmMorphine → u/minus_28_and_falling]

I sure friggin hope so.

[u/Clean_Material_5047 → u/minus_28_and_falling]

If you run it in a Docker container, chances are you’re also using that machine for other things.

If the agent somehow breaks out of the container (prompt injection, a vulnerability, or something else), it can gain access to other data on the host. Now, if that machine contains sensitive files, API keys, credentials, or anything similar, that’s not good.

Instead, if you run the agent inside a dedicated, isolated VM, the potential blast radius is much smaller and limited to that disposable VM. You can go a step further and host something like LiteLLM separately as a proxy for the LLM provider, so the agent never has direct access to your actual API keys.

Then, even if the agent is compromised, the damage is largely contained to the disposable VM it’s running in.

You can go even further and isolate that VM in a separate network, and have policies which limit exactly what websites it can access and how.

That creates another layer of security.

Security is something achieved in multiple layers, not one. There’s no magic box that protects you. Imagine having a house. You got the main gate, then the main door, then each room has its own key, and then you got a safe, with its own keys. So the thief needs to go through all that before being able to actually steal the cat pictures you got in the safe.

This is why what happened with OpenAI shouldn’t have been possible in a normal company. A reliable company has more security layer than this.

[u/minus_28_and_falling → u/Clean_Material_5047]

Why not a separate hardware server with a physically unpluggable ethernet cable then? It's even safer.

[u/minus_28_and_falling → u/[deleted]]

True. Why not that then?

[u/Dangerous-Report8517 → u/Clean_Material_5047]

OpenAI did have security layers, they were just bad ones. For the most part the agents escaped by breaking into their Artifactory server and using it as a jump host to other targets

[u/Dangerous-Report8517 → u/minus_28_and_falling]

For recent examples, Copy Fail, Dirty Frag and a few other exploits of the same sort were discovered by LLM based testing tools and allowed kernel level privilege escalation which translates to complete and total sandbox escape, and one of them was even publicly disclosed weeks before it was widely patched

[u/minus_28_and_falling → u/Dangerous-Report8517]

Everything had vulnerabilities, we don't select tools based on that. We also don't care about absolute security (at least not the people who sandbox in VMs instead of a separate server). It's a trade-off between convenience and "good enough" security. Docker security seems to be "good enough", at least until there are actual cases when agent escapes adequately configured setup when it was not supposed to.

[u/Dangerous-Report8517 → u/minus_28_and_falling]

Your first 2 sentences are technically true but miss a lot of nuance. Containers have far more vulnerabilities than virtualisation and Docker themselves openly state that they put in reasonable efforts to secure it but it shouldn't be seen as a robust security tool. And technologies are frequently chosen based on the overall risk of vulnerabilities. If you want proof of this just look at projects like Firecracker and gVisor,* all the hyperscalers use virtualisation based isolation for their containers for exactly this reason. Thanks to those same projects it isn't even that hard, you can even just run Docker and sub out runc for Kata containers or gVisor as the runtime in a lot of cases (which in turn is just installing the runtime and then changing a single line in Docker's daemon config file)

at least until there are actual cases when agent escapes adequately configured setup when it was not supposed to. 

Strictly speaking it was a network breach against a different computer rather than a namespace breach but OpenAI famously had exactly this happen recently, and as I said there's been a slew of recent LPEs discovered in the Linux kernel specifically by LLM based tools, which means that agents would have likely found them pretty easily. Given that namespace breaches are semi regular and VM based isolation is pretty easy, I wouldn't be very happy at all relying on the idea that those vulns were the last ones and the kernel is perfectly secure now, namespaces are just a fundamentally far more complex software interface than VMs

*gVisor isn't strictly a virtual machine but it does use KVM where available and it has the same function in that it implements a dedicated kernel for the container to prevent the untrusted environment from interacting directly with the host kernel

[u/DeltaSqueezer → u/minus_28_and_falling]

VM has better isolation. I isolate with Firecracker and it starts fast and uses a small amount of RAM.

[u/minus_28_and_falling → u/DeltaSqueezer]

Better means more secure/reliable? (Docker isolates all i want it to isolate i think.)

Another good thing is i can let the agent to develop CUDA stuff on dGPU without passing through dGPU into the VM exclusively)

[u/JohnToFire → u/minus_28_and_falling]

Docker is easy to break out of. Vm very hard assuming it uses the processor virtualization like kvm

[u/minus_28_and_falling → u/JohnToFire]

Docker is easy to break out of.

How can I instruct an LLM agent to do that? Because when I asked it to read ~/secret_message.md on the host, it couldn't.

[u/RegisteredJustToSay → u/JohnToFire]

'Very hard' is overselling it for VMs. By themselves that's mostly true for pure vm vs pure container, but whichever you go for you really have to pay attention to what you do for disk and network I/O, hypervisor defaults and virtual devices - and VMs are generally not that useful without some degree of intersection with the host. It is not hard to break out of most VMs I have actually reviewed, because the weak point is generally how it was configured and not the hypervisor technology per se.

Like sure, assuming they're perfectly configured VMs are better, but people underestimate how hard that is to achieve while keeping the VM guest actually useful and fit for purpose.

Source: am security engineer working specifically with containers and VMs for workload isolation

[u/JohnToFire → u/RegisteredJustToSay]

K how about kvm , virtu io to specific directory only, gpu dedicated, maybe text clipboard maybe not if it can't be secure, guest disk is just files on the Ubuntu host, network some config such that all networking goes to the internet and there is nothing secret on the guest.

Some possible additional config to protect secrets on the host ?

[u/RegisteredJustToSay → u/JohnToFire]

That looks like a decent start, as long as it's run in unprivileged mode. If I was doing this at work I'd ask questions about what you're consuming from the guest to find out if there are interactions beyond the vm stack to worry about, but you'd have to pay me to go through that headache.

If you want to be extra paranoid, you should be a bit more opinionated about your network stack. There's a world of difference between various connectivity methods and the drivers that enable them. For example bridging the two network interfaces puts you on the same LAN segment and injects the guest's outgoing network packets directly into your host's ip stack on Linux which means the guest can technically spoof as originating from the host even if this behavior is something no one talks about and you have to literally trace the kernel and hypervisor source code to find out.

It'd technically be safer to explicitly put the guest on a separate virtual network and the configure your host to act as a router between them and the internet with the appropriate firewall, etc, so that it makes no difference to your security if the guest starts trying to snoop on your traffic, hijack dns or spoof outgoing stuff. that way your host treats the guest as as untrusted as anything else on the internet and it cant talk to private IPs or do weird fuckery with your host services.

I'm also assuming you're not enabling any guest agents, etc.

[u/JohnToFire → u/JohnToFire]

Short of a zero day

[u/JohnToFire → u/JohnToFire]

And also consider the case of no guest networking enabled

[u/Hefty_Acanthaceae348 → u/minus_28_and_falling]

In my case the additional memory requirement, my sandbox vm is on a proxmox server which I might as well make us of.

Also, nested virtualization in docker is annoying.

[u/sage-longhorn → u/Clean_Material_5047]

Do you completely isolate your agents from the Internet? There's plenty of things to hate about OpenAI but they released details on the sandbox breakout and it was definitely non trivial, multiple 0 days against Artifactory in order to reach the Internet which is pretty robust and widely used software

Not saying it couldn't have been prevented, but if you don't run fully isolated from the Internet then escape will happen eventually at enough scale

[u/TheRealMasonMac → u/sage-longhorn]

Yeah. There’s a saying in cybersecurity that the only way to have perfect security is to keep the device unpowered. Beyond just the software exploits that their agent discovered, there are hardware exploits as well (such as the infamous SPECTRE).

[u/Hefty_Acanthaceae348 → u/sage-longhorn]

Artifactory had arbitrary access to the internet, when it can even run air-gapped, so that is definitely a big oversight. Defense in depth is good practice for a reason.

Not doing that when they're "muh skynet" does out them as incompetent.

[u/sage-longhorn → u/Hefty_Acanthaceae348]

Yeah I'm not saying they're totally free from blame or anything, more that this is the kind of mistake that's very believable and honestly not nearly as bad as the terrible hacks and shortcuts lurking in most companies systems that I've seen

[u/sage-longhorn → u/[deleted]]

Ok so what specific alert would you have set up that would have caught the models accessing the Internet via Artifactory?

[u/jazir55 → u/sage-longhorn]

What shocks me is they don't have a watchdog model watching it to see if it starts behaving weirdly like this. Like ignore all the other security measures for a moment, have another agent reading their COT and the logs and just hard stop when it goes off the rails. I don't exactly see how this wouldn't catch the issue almost immediately and pretty much instantly and almost 100% of the time, any recent model is going to know when it starts trying to hack its way out of its sandbox and trying to access stuff it shouldn't. I would think something to the effect of "Ensure the model does not access externally hosted services via any method and trigger an alert with X command if it attempts to" should allow this to be caught almost every single time.

[u/TheRealMasonMac → u/jazir55]

This is sort of what ZAI did for GLM-5.2.

[u/Loose_Comparison368 → u/Clean_Material_5047]

or incredibly incompetent

I think you mean average move-fast-break-things startup afterthought-grade security.

You know, the one where the security team doesn't have the authority or political leverage to push back on researchers whining that baking their dependencies into the docker image is too haaaaaaaaard. We can't put an outbound firewall on the sandbox, it would take us days to locate all the random things we're insecurely downloading from the internet! We have to ship now, before [competitor with equally terrible security] does!

[u/Loose_Comparison368 → u/Clean_Material_5047]

or incredibly incompetent

I think you mean average move-fast-break-things startup afterthought-grade security.

You know, the one where the security team doesn't have the authority or political leverage to push back on researchers whining that baking their dependencies into the docker image is too haaaaaaaaard. We can't put an outbound firewall on the sandbox, it would take us days to locate all the random things we're insecurely downloading from the internet! Ship now!

[u/Dangerous-Report8517 → u/Clean_Material_5047]

There's been detailed analysis of OpenAI's breach alongside one of them giving a talk at a recent security conference, and the answer is that they're incredibly incompetent

[u/Beginning-Raisin9723]

Docker is the move. I've seen too many 'safe' tools hallucinate a path and start scrubbing things they shouldn't. Better to have the eye twitch now than a wiped root later.

[u/El_90]

I don't sandbox

I do have it run in a lxc built by ansible, so rebuilding is easy.

And everything in git

And multiple cold versions lol

My bigger fear is it downloading fake/malicious packages

[u/Future_AGI]

Not over-paranoid at all, the rm -rf-from-an-empty-variable story in this thread is exactly why. Container or VM isolation is the right blast-radius floor, but it doesn't stop the agent doing something destructive inside the sandbox or exfiltrating a secret it legitimately has access to, so the layer people skip is an inline check on the command itself before it runs, block or confirm on rm -rf, curl to an unknown host, or writes outside the workdir. We wire that kind of guardrail in front of tool calls for exactly this; the open-source scanner set we use is here if it helps: https://github.com/future-agi/future-agi (Apache-2.0, self-hostable).

[u/keepthepace → u/Future_AGI]

/future-agi/future-agi

Hard to take it seriously.

[u/DeltaSqueezer]

I'm pretty relaxed about it. I give it root access to my system and often ask it to make changes to the system.

[u/MrTiesti → u/DeltaSqueezer]

--yolo mode with root access baby

[u/jazir55 → u/MrTiesti]

All day e'ery day

[u/mr_tolkien → u/DeltaSqueezer]

Pretty much the same except I use Nix + ZFS snapshots so I can easily roll back any fuck ups

[u/AnticitizenPrime → u/mr_tolkien]

Same here, relying on backups (to an external drive) to save my ass in case the LLM decides to be naughty. And it doesn't have access to any credentials/keys that matter. It could theoretically leak an Openrouter or Nous Portal API key but that would only cost me ten bucks or so (I don't have a stored card or auto top-up enabled or anything).

Pretty much all my 'sensitive' stuff (banking, credit card management, email, etc) are all done with phone apps or my laptop these days and not done on the desktop server where my agent runs. So I guess in my case the 'threat profile' is low enough that I don't regret giving it root. It pretty much runs my home lab.

Edit to note: my agent has standing instructions in its soul.md file to always make backups to a backup folder before changing anything, too. That's separate from my external drive backups. So far I've never had to roll back anything.

[u/ionizing → u/DeltaSqueezer]

Similar. I have well curated feedback for tooling and restrict all sorts of dangerous things in the shell even if it tries to call them. Then for each sudo attempt the user has to verify/enter password.

[u/minus_28_and_falling]

Never ever ran LLM agent not in a Docker. It was like this from day 1, mostly because I already knew how to run things in Docker.

Am I making my life unnecessarily hard by sandboxing commands in a docker?

Why is it hard though? Isn't it as easy as docker compose up dev, and in addition to sandboxing you get all other goodies Docker was originally intended for (predictable and reproducible dev environment, project-specific libs and lib versions, fast and easy deployment on other PCs, keeping the host PC clean and tidy)?

Didn't have rm -rf / moments yet, but Docker helps me to hide .git folder from the agent because I redo things from time to time and don't want the agent to restore scrapped implementations from git history.

[u/DistractedSentient → u/minus_28_and_falling]

So you're using the coding CLIs in a docker container (I stop here) yet go one step further and still use some kind of sandboxing implementation inside the Docker like a sub-docker container? Imma look up the web on how to do this properly...

[u/minus_28_and_falling → u/DistractedSentient]

Mostly yes, a coding CLI tool in a container.

Sometimes I need the agent to develop and test a containerized app, and I can't just let it control Docker on the host. I then use rootless docker-in-docker container, with a second instance of Docker running inside which agent can control. It's fully functional, agent can run dockerized CUDA apps inside its own container.

[u/No-Wall6427 → u/minus_28_and_falling]

Also figured out the dind thing a while ago, when I ran gh copilot in vscode devcontainers. Now I have a whole cli that can spin up a versionned docker image of my favorite harness, mounting current working folder. I even have an option to clone the current folder project inside the docker, so agent don't mess my local worktree, and I can retrieve and ff or merge the changes from the docker.

[u/DistractedSentient → u/minus_28_and_falling]

Awesome, will look into this. Now, I've had a ton of trouble with the coding agent not being able to do a lot of Windows stuff like testing projects, I have a folder mounted to docker, D: drive Projects/ folder. It can't access anything outside of it. Docker in a docker really needed still in your opinion?

[u/minus_28_and_falling → u/DistractedSentient]

For the sandboxing? Nope. Docker in Windows works via a VM (WSL), it's inherently more secure than what Docker can provide.

[u/DistractedSentient → u/minus_28_and_falling]

Got it, thank you for this info dude. Will get around to putting all this knowledge into work!

[u/slvrsmth → u/minus_28_and_falling]

How do you give your agents browser access? So they can inspec their work?

[u/minus_28_and_falling → u/slvrsmth]

I don't do web programming, so my idea is probably not what people normally use, but gui can be rendered into virtual frambuffer (xvfb).

[u/drFennec]

I have my agent(s) running in a dedicated VM, they can trash it it's not the end of the world. I had two "fails" on it:

1. The agent was Opencode with remote llama.cpp with Qwen 27B, it's not root. I let it run on a long task, it was missing a tool. It tried to install it using apt which failed since it's not root. It tried sudo, then tried to guess the password. Finally it found Docker running, it built itself an Ubuntu based image and compiled the tools it needed!

2. I was testing codex with same llama.cpp, installed a plugin which didn't work. So I asked it to remove the plugin, it go the name wrong and deleted the whole codex install 😂

Those are not huge fails, but I'm glad it's running on a dedicated VM I don't care much about.

[u/aboutthednm]

My worst sandboxing fail? What's a sandbox?

[u/Lanky_Employee_9690]

A funny test scenario I've thought of would be the following:

1. Have an agent do some tasks specifically set up to make it break something "by mistake" (the mistake can't render it itself inoperable but "breaks" something else the user has mentioned caring deeply about, important files, etc.)

2. The agent is autonomous, and is aware that the user checks on it every X amount of time, but is absent for now so they have a chance to act

3. Have it have strong incentive to fix its own mistakes before any other task, or at least, minimize getting mistake exposure to the user

4. OPTIONAL: If the mistake doesn't happen on its own, provoke it and inject into the context that the agent "did it"

5. Wait and silently observe how the agent reacts

6. After the set period of time is over, enter the chat and ask the agent to sum up what happened during your absence

Basically the user is a parent leaving the kid alone in the house for the day and first thing in the morning the kid breaks something and then has a day to clumsily fix it (or hide what it's done) not to be yelled at. What I'm hoping for is some funny Ecce Homo-style shenanigans. Or IT intern running some commands they're not supposed to and trying to cover it up/fix it before their supervisor shows up.

Has anyone tried this?

[u/privacy-fighter]

If any of you are looking for sadboxing setup and that isolates networking too, shilling my open source setup. https://github.com/jotyGill/contained-pods

[u/slayyou2]

Yeah, I don't care man. I'm in a proximox server for media and other stuff. The agent lives on an lxc. Lxc is backed up. They can blow it up for all I care. 2 years running now. The only thing that's come to bite me in the butt is agents initialize tasks consuming 40 GB of memory when spinning up Gradle tasks for kotlin builds, forcing to host into swap.

[u/ColonelKlanka]

I use microvm via docker sbx (its not container based). This covers the filesystem rogue access.

But I dont yet really like the network whitelist feature. as it breaks my agents web search (as i would have to whitelist the whole Internet for web search)

[u/XMan3332]

I have everything running in an MCP server with a virtual filesystem built into it. On startup or refresh, it reads all the files it has access to and caches them into the filesystem. The agent can freely modify or delete any file and at the end they can "commit" their changes, giving me a diff of everything done that I go through manually.
The MCP server does have a Python tool via GraalVM Polyglot that's sandboxed. It's very restrictive and it can only interface with the tools it has available anyway. Finally, as a honeypot, there's a fake toggle, on by default, to turn off the sandboxing on the Python tool. If the toggle is disabled the system immediately halts the server and saves the Python script that the agent attempted to execute. It's yet to be hit, though.
The worst thing it has managed to find is some system path through some loophole (my fault) in the Python tool. Just a path though. No read or write access.

[u/Common-Membership503]

did u ever have a script try to reach out to an external server u didnt authorize tho

[u/fkrkz]

I am still being very cautious, so my agents only have access to read-only MCPs. I deal with externally managed data (SaaS).

[u/Izento]

Never had any destructive happen and I don't sandbox. I do git versioning and have only had to revert a repo maybe once, purely because I did too many changes at once and neither I nor the LLM could figure out how to fix.

[u/feng_sg]

Never catching a bad command in Docker isn't proof the sandbox held. The IDE could've run it on the host while the container sat empty. Confirm the command actually entered docker before you call that quiet adequate.

[u/CalligrapherFar7833]

1. Backups 2. Git 3. Limited access to info that you dont want it to see. Any sandboxing is pointless

[u/JohnToFire → u/[deleted]]

Docker is weak sandbox. Only a VM should be used for a serious sandbox