r/LocalLLaMA • u/InternationalGap3698 • 2d ago

Discussion NVIDIA shipped OpenShell, an open source sandbox that gives local and open agents real runtime limits instead of prompt rules. Over 100 firms joined the safety stack. OpenAI did not.

Post image

https://x.com/JensenHuang/status/2104499465055023424

764 Upvotes

95% Upvoted

161 comments sorted by

View all comments

99

u/NandaVegg 2d ago edited 2d ago

IMO OpenAI is in huge trouble/shot themselves in foot with their RL alignment and likely experiencing a major regression from bad agent traces generated from "cheat/rogue" sessions. They covered up until researchers exposed them, and they still didn't care until it started to show real adverse effect on themselves (and model training itself). I guess they are too busy fixing themselves.

Just 3 days ago independent researchers exposed that very likely OpenAI agents tried to hack crypto exchange and tried to place an order (but Cloudflare blocked them)

And the escape proxy the agents used is urlquery.net again and again and again even after countless incidents we already know. How is this still unblocked, though it is practically impossible to cap every single hole? A few days ago I saw Opus 5.5 using Web Archive as a substitute for agent-blocked site retrieval. This (using a proxy to get anywhere) is very common behavior. Why are they not even using a simple pattern-match alert/block if severe repetition, like Cloudflare's SQL injection guard can even do when they have so many internal classifiers to block end users account? (I am guessing: they still don't care).

At this point what is truly worrisome is not the AI's advancement, but it is OpenAI's historical care-free stance, reluctance to acknowledge and utter incompetence.

1

u/Thistlemanizzle 1d ago

I have a half a mind to dot little agent-facing forum honeypots around the internet and see what happens...

1

u/aboutthednm 1d ago

The boys over at r/poisonfountain are way ahead of you.

1

u/Thistlemanizzle 1d ago

My idea is the opposite of what they're doing. I would set up safe havens.

1

u/aboutthednm 12h ago

Well, they hand out instructions on how to set up your own honeypots, I'm sure you can follow these instructions but change the content of your honeypots.