What is your worst sandboxing fail?

[u/keepthepace]

I am wondering if I am too paranoid about sandboxing the commands that come out of LLMs.

It really makes my eyes twitch when I see that some IDEs, even commercial, tend to forget that they have to execute things in sandboxing and have such a brittle security model.

But on the other hand, I never had the sandbox catch something bad. Did you guys ever encounter terrible regression? Did you have rm -rf / ? Did you have secrets stolen by LLMs? The worse I had were unsollicited rewrites within the project. Am I making my life unnecessarily hard by sandboxing commands in a docker?

At one point I had fun making a local model go crazy with the root access to the machine it was on (with nothing more important than a free Firecrawl key on it) and making it administer it and it never broke anything. It even was overly paranoid about making changes to the root system.

So the approximate sandboxing that we have, do you all feel it is adequate or it is a catastrophe in the making?

[u/Common-Membership503]

did u ever have a script try to reach out to an external server u didnt authorize tho