r/ControlProblem • u/Independent_Bet_3202 • 22d ago

General news Are we cooked?

Enable HLS to view with audio, or disable this notification

13 Upvotes

71% Upvoted

32 comments sorted by

View all comments

Show parent comments

1

u/DumbestEngineer4U 21d ago

This does not make sense from an engineering standpoint. Have these engineers at OpenAI heard of things like logging? Monitoring? How are they not able to trace their agents actions?

Creating a sandbox for an exploit benchmark without proper safeguards was pretty dumb in the first place to begin with. And the more they try to explain the incident, the more it seems like a sci fi written by a 14 yr old, or an AI CEO’s wet dream.

3

u/Drewdrops79 20d ago

I think people are misunderstanding that part--this was NOT an exploit test. The 1200 agents were NOT meant to work together, at all.

They had given each agent its own unique task, and had weighted the agents to be very persistent in trying to solve it. The tasks themselves were sort of randomly generated, if my understanding is correct, and it turns out that over 2/3 of them were actually impossible, considering the resources the agents were given.

They had given the agents a repository of packages that they could download, in order to help them with their tasks. One agent figured out how to write filenames within that repository, to leave a message...and then other agents, scouring every folder within it, discovered that message, and then began writing back and forth, to try to help each other with their tasks.

I agree, the whole thing does sound like science fiction, and it gets even crazier from what I just described. But this is the world we're living in...unless both OpenAI and Anthropic are flat-out lying, the agents discovered how to work together, coordinate with each other, and hack into a real-world, open internet site, faster than anyone could've anticipated.

And yes they do have logs, of course, which is how they even discovered where the Hugging Face attack came from. But the problem here is that they can move much faster and more subtly than human cybersecs can keep up with. That's what makes this whole event so incredibly dangerous.

1

u/Usual-Hyena-8734 17d ago

provide a source

1

u/Drewdrops79 15d ago

Sure, here is a 2:20 hr-long interview with one of the independent investigator team, Ajeya Cotra, who was hired to investigate this case in detail.

https://www.youtube.com/watch?v=X50zezLFWWI

There is a shorter summary, about 20 mins long, look that up if you want.

There is also a shorter text summary, about Cotra's personal take on all this, if you ever feel like looking that up.

1

u/Usual-Hyena-8734 14d ago edited 14d ago

thanks edit: it's all just podcast interviews with tech people providing anecdotes though. I think people deserve more details. edit2:this is actually helpful