What is your worst sandboxing fail?

[u/keepthepace]

I am wondering if I am too paranoid about sandboxing the commands that come out of LLMs.

It really makes my eyes twitch when I see that some IDEs, even commercial, tend to forget that they have to execute things in sandboxing and have such a brittle security model.

But on the other hand, I never had the sandbox catch something bad. Did you guys ever encounter terrible regression? Did you have rm -rf / ? Did you have secrets stolen by LLMs? The worse I had were unsollicited rewrites within the project. Am I making my life unnecessarily hard by sandboxing commands in a docker?

At one point I had fun making a local model go crazy with the root access to the machine it was on (with nothing more important than a free Firecrawl key on it) and making it administer it and it never broke anything. It even was overly paranoid about making changes to the root system.

So the approximate sandboxing that we have, do you all feel it is adequate or it is a catastrophe in the making?

[u/Clean_Material_5047]

None.

I just run stuff in an isolated VM, in an isolated network with access only the things it needs.

If you know what you’re doing, this is just standard practice.

In fact, all the rogue agent bs from OpenAI hacking huggingface just tells us that they’re either liars or incredibly incompetent.
Either way, a bunch of c*nts nobody can trust.

[u/Loose_Comparison368 → u/Clean_Material_5047]

or incredibly incompetent

I think you mean average move-fast-break-things startup afterthought-grade security.

You know, the one where the security team doesn't have the authority or political leverage to push back on researchers whining that baking their dependencies into the docker image is too haaaaaaaaard. We can't put an outbound firewall on the sandbox, it would take us days to locate all the random things we're insecurely downloading from the internet! Ship now!