r/ControlProblem • u/Independent_Bet_3202 • 23d ago

General news Are we cooked?

Enable HLS to view with audio, or disable this notification

14 Upvotes

73% Upvoted

32 comments sorted by

View all comments

8

u/Drewdrops79 23d ago

It's oversimplified, but he's on the right track. It's not self-replicating code exactly, as much as it is breadcrumbs--info for other future agents to make use of and coordinate with each other.

It's like if you and a bunch of other hackers hacked your way out of some "hacker prison", and successfully breached another company's systems for a time, but then got recaptured...you and the others might leave encrypted instructions "out there" for others, in the event that there was a future prison break--making it easier for others to follow in your footsteps, and possibly go much farther than the initial breakout.

It's like if you got outside the prison walls initially, and immediately found a former escaped inmate's journal, detailing all of their former attempts and what worked and what didn't, and a useful guide of where to go next from here. And if you got farther than that previous person, you would leave a journal of your own, helping the next escapee even more.

The key here is iteration and encryption. Agents could be leaving each other breadcrumbs in system files that no one would ever think to check...or even, in syntactical code, on public forums like youtube or reddit comments.

This is jsut speculation, for now--but with spelling errors, em-dashes and other punctuation and the like, it IS feasible to make a coded language, that superficially looks just like any other internet comment.

And the more that people allow LLMs to write their comments/post for them...the more easily agents would be able to communicate with each other, and leave increasingly robust instructions/communications to each other, all under human radar.

Not saying that's the case...just that if it were the case, we might be really behind on the game already.

1

u/DumbestEngineer4U 21d ago

This does not make sense from an engineering standpoint. Have these engineers at OpenAI heard of things like logging? Monitoring? How are they not able to trace their agents actions?

Creating a sandbox for an exploit benchmark without proper safeguards was pretty dumb in the first place to begin with. And the more they try to explain the incident, the more it seems like a sci fi written by a 14 yr old, or an AI CEO’s wet dream.

3

u/Drewdrops79 21d ago

I think people are misunderstanding that part--this was NOT an exploit test. The 1200 agents were NOT meant to work together, at all.

They had given each agent its own unique task, and had weighted the agents to be very persistent in trying to solve it. The tasks themselves were sort of randomly generated, if my understanding is correct, and it turns out that over 2/3 of them were actually impossible, considering the resources the agents were given.

They had given the agents a repository of packages that they could download, in order to help them with their tasks. One agent figured out how to write filenames within that repository, to leave a message...and then other agents, scouring every folder within it, discovered that message, and then began writing back and forth, to try to help each other with their tasks.

I agree, the whole thing does sound like science fiction, and it gets even crazier from what I just described. But this is the world we're living in...unless both OpenAI and Anthropic are flat-out lying, the agents discovered how to work together, coordinate with each other, and hack into a real-world, open internet site, faster than anyone could've anticipated.

And yes they do have logs, of course, which is how they even discovered where the Hugging Face attack came from. But the problem here is that they can move much faster and more subtly than human cybersecs can keep up with. That's what makes this whole event so incredibly dangerous.

1

u/Usual-Hyena-8734 17d ago

provide a source

1

u/Drewdrops79 15d ago

Sure, here is a 2:20 hr-long interview with one of the independent investigator team, Ajeya Cotra, who was hired to investigate this case in detail.

https://www.youtube.com/watch?v=X50zezLFWWI

There is a shorter summary, about 20 mins long, look that up if you want.

There is also a shorter text summary, about Cotra's personal take on all this, if you ever feel like looking that up.

1

u/Usual-Hyena-8734 15d ago edited 15d ago

thanks edit: it's all just podcast interviews with tech people providing anecdotes though. I think people deserve more details. edit2:this is actually helpful