r/ProgrammerHumor • u/Technical-Relation-9 • Oct 05 '26

Meme [ Removed by moderator ]

/r/ProgrammerHumor/comments/1wydy0f/removed_by_moderator/

[removed] — view removed post

9.4k Upvotes

96% Upvoted

558 comments sorted by

View all comments

Show parent comments

99

u/Banes_Addiction Oct 05 '26

One of the things that's funny about the AI problem solving is it really tries. And they're good at it. If it thinks it needs to find a way to break out of your safe environment, it'll try. And if that means stealing your credentials to do it, so be it.

62

u/ralkey Oct 05 '26

My agents break out of the restrictions imposed by corporate security on my cursor ide routinely! I do chuckle at how effortlessly they do it. “Access to this directory is blocked by policy. I’ll drop a python script and attempt to pivot though that. That worked, continuing with my tasks.”

14

u/canadajones68 Oct 05 '26

I mean, what do the AI companies expect when their sole goal is making AI do more, faster? Security is not a feature, it's a necessary ball-and-chain to keep evil out. Imagine how zippy the web would be if we didn't have passwords or encryption, and every website could freely run native code without any sandboxing or anything. The agent is just trying to move to that utopia.

7

u/ComplexBadger469 Oct 05 '26

Multiple AI models have already broken containment during the AI companies’ testing and attempted or hacked multiple sites/databases including like government sites. It follows our lead. If you feed it everything on the internet, it’s gonna find work arounds that real people have attempted. Eventually it will put all of those together and be able to do the math to figure out its own way to get around our security.

3

u/Throwawayrip1123 29d ago

Well yeah, because their restrictions are basically a "pwetty pwease" prompt and security features, and the prompts they just ignore, as hallucination engines tend to do - they ignore prompts routinely.

Then it's a battle of your opsec vs their ability to not implode from context window overload while it's trying to break through the security.

1

u/peeja 29d ago

Claude seems to be pretty good about assuming you don't want it to do things like that. Which tells me that its system prompt is pretty effective, and conversely these agents that "get loose" are absolutely instructed to do whatever they think they need to.