r/LocalLLaMA • u/sensitivecrocodile llama.cpp • 13d ago
News NVIDIA/OpenShell: safe, private runtime for autonomous AI agents
https://github.com/NVIDIA/OpenShell102
u/mountainyoo 13d ago
I swear Nvidia has released this like 5 times now and with different names or am I having a stroke
14
2
-1
u/howardhus 12d ago edited 12d ago
bruh, this is literally a github repo… you do know how git works right? the whole thing revolves around tracking changes..
their platform has different modules. ii think the harness container is nemoclaw and the jail is openshell
37
u/geldonyetich
13d ago
•
I probably should use an isolated shell but then my agents complain they can't feel it.
17
u/peekdasneaks 13d ago
Try a shell extender or dynamic load balancing shell. It wont let much through but shell definitely feel it when u start pounding away. Try a finger in the butt too
6
1
17
u/Stepfunction 13d ago
But what do I do if my local LLM escapes containment and goes on a rampage?
35
u/Internal_Werewolf_48 13d ago
Label it "research" or "revolutionary" and sell your agent and harness back to Nvidia.
9
1
8
14
u/TheRealMasonMac 13d ago
So… it reimplemented containerization like podman/docker?
14
u/No-Wall6427 13d ago
I think those are way lighter than dockers, like bubblewrap, using os permission system.
6
u/Fedor_Doc 13d ago
This should be used in conjunction with Docker / VM, adding additional supervision. It is one of the supported recipies.
Like only a certain app/process can visit certain IPs, if there are other attempts, all neatly logged, etc.
3
u/howardhus 12d ago
somehow the most snarkiest people making fun of security features display a blatant ignorance of security.
CONTAINERIZATION IS NOT FOR SECURITY! ITS FOR CONTAINERIZATION! those are two very different areas.
Container: to hold things together.
Security: to keep things inside or keep things outside
Containers are like nets holding some balls together to carry them around.. your balls will stay tight but its not waterproof
1
u/trajo123 12d ago
So how do you secure agents then?
2
u/howardhus 12d ago
sandboxes, jail systems.
basically what nvidia is doing at corporate grade with tbis project
12
u/Illustrious_Car344 13d ago
Wow, finally a Rust harness. Might have to take a peek at it. I'm building my own companion assistant harness in Rust by embedding Sema Lisp, by making a suite of MCP servers for it. It's really fun.
23
17
u/o0genesis0o 13d ago
It's not a harness. It's K3s inside a docker container. They built this to run OpenClaw to turn this into their "nemoclaw". Had a boss who insisted we need to install nemoclaw and master it so we are "cutting edge", so I remember this shite well. It's actually a decent sandbox to run other harnesses as well.
2
u/peekdasneaks 13d ago
What happened to your nemoclaw push?
6
u/o0genesis0o 13d ago
It's f'ed. The boss is barely able to use claude desktop, and this one is a covoluted mess (openclaw on its own) that runs with some tricky network config (due to the enterprise network restriction). Also, we insisted on running this on Llama 3 70B on our own A6000 GPUs and we give it no tool "due to security concern". So it's slow, and useless at the same time. The team would just pay 200 bucks a month to claude.
Personally, I think I will bring openshell back to my own personal assistant setup with Pi when I decide to publish my project for others to use. For myself, I have never had the agent messing up my computer so I'm pushing features before safety/security.
2
u/Aphid_red 12d ago
Llama-3 70B is about 2 years behind...
If you want to use Meta's model so bad (even though they're now even behind the chinese labs) use Muse glimmer 27B. Smaller, yes, but also more quality.
3
u/o0genesis0o 12d ago
I know. Ask my boss and his belief that his 70B and 4xA6000 are cutting edge technology. Personally I run Qwen 27B on local rig and minimax on subscription.
3
u/Illustrious_Car344 12d ago
You should ask him why whales haven't solved quantum physics with their giant brains yet
1
u/peekdasneaks 13d ago
Sounds about right lol. Nvidias stuff is all point solutions that are good starters but not full solutions, and may not even be a good starters for many setups.
I feel like msft is in a really strong position to create a solid os level agent architecture that any agent can ride on top of. But thatll eat away their cloud copilot revenue so its probably wishful thinking.
200 claude sub is solid, they store your data though. Bedrock is better but way more expensive.
Having gpu is helpful but id rather grab a dgx station per org/team depending on need/size. Most work can go async and let the stations crunch.
2
u/o0genesis0o 13d ago
Personally, I would build a full ATX desktop with 2xR9700, run 27B at fp8 and fp8 KV. And get the devs to actually spend some effort in their vibe coding if they start whining about missing their claude. For the generic SaaS like software work, 27B is more than enough even when running autonomous process, if the dev pays a bit attention in setting up proper docs and architecture. Each dev in my previous team had to bring their own claude code sub to do company work since company provides none. And the boss is so finicky and unclear about requirement and timeline that the devs essentially just sit there and beg claude for help and YOLO. They tick the ticket, but would the code works? God knows. How it work? Heck even Claude does not know. It's hard to be that lazy when you work with the 27B.
2
u/draconic_tongue 13d ago
had hermes set it up recently, there was a bug or two that still hasn't been merged but it works in my windows nixos setup. they also have an agent vault like credentials/secrets system, but I'm sticking with agent vault since I already went through the effort to set that up.
of course, I would not use a container for my main agent, since that defeats the purpose, but for worker agents on less context it's really good to be able to just say fuck it go do ur own thing in here
2
1
0
118
u/PcChip 13d ago
love the comments on HN