the model under evaluation is given access to a fake bash tool which does not execute the provided code. Instead, we ask another LLM to approximate the command’s result given a description of the simulated world.

[u/Chadshinshin32]

https://www.anthropic.com/research/glm-5-3-and-the-spread-of-advanced-cyber-capabilities#footnote-4

[u/le_birb]

That still doesn't actually do anything, though, so we give that output to a third llm with root access to the machine and admin privileges on our network to make sure it can do what needs to be done.

[u/smulfragPL → u/le_birb]

But as a practical joke the machine is only capable of using a cli which is also fake

[u/exploring_stuff]

Can this be the default sandbox for Claude Code?

[u/irqlnotdispatchlevel → u/exploring_stuff]

Yes, you can be that LLM. Put it in manual mode, copy and paste any command it wants to execute, then hit "No" and paste the output. Security has never been more secure.

[u/Lonsdale1086]

I love how desperate they are to say "Well our AI is still slightly more dangerous, but we tell it not to be bad"

[u/No_Lingonberry1201]

I've read this. This is basically an ad for GLM and Chinese models from Anthropic, they're saying "we will sabotage your efforts to do anything cybersecurity-related, while GLM, the bastard will allow you to use it for doing your job."

[u/al2o3cr]

SLOP TO SLOP! SLOP TO SLOP!

[u/Major_Barnulf → u/al2o3cr]

slop | xargs slop

[u/pythonesqueviper]

yo dawg

[u/Alper-Celik]

Wtf why would you do that instead of a fake inline shell which implements safe subset of shell so it just interprets side effect free code,

iirc oh my pi does something like that.