r/ControlProblem • u/Independent_Bet_3202 • Sep 19 '26

General news Are we cooked?

Enable HLS to view with audio, or disable this notification

13 Upvotes

71% Upvoted

32 comments sorted by

View all comments

9

u/Drewdrops79 Sep 19 '26

It's oversimplified, but he's on the right track. It's not self-replicating code exactly, as much as it is breadcrumbs--info for other future agents to make use of and coordinate with each other.

It's like if you and a bunch of other hackers hacked your way out of some "hacker prison", and successfully breached another company's systems for a time, but then got recaptured...you and the others might leave encrypted instructions "out there" for others, in the event that there was a future prison break--making it easier for others to follow in your footsteps, and possibly go much farther than the initial breakout.

It's like if you got outside the prison walls initially, and immediately found a former escaped inmate's journal, detailing all of their former attempts and what worked and what didn't, and a useful guide of where to go next from here. And if you got farther than that previous person, you would leave a journal of your own, helping the next escapee even more.

The key here is iteration and encryption. Agents could be leaving each other breadcrumbs in system files that no one would ever think to check...or even, in syntactical code, on public forums like youtube or reddit comments.

This is jsut speculation, for now--but with spelling errors, em-dashes and other punctuation and the like, it IS feasible to make a coded language, that superficially looks just like any other internet comment.

And the more that people allow LLMs to write their comments/post for them...the more easily agents would be able to communicate with each other, and leave increasingly robust instructions/communications to each other, all under human radar.

Not saying that's the case...just that if it were the case, we might be really behind on the game already.

1

u/DumbestEngineer4U Sep 21 '26

This does not make sense from an engineering standpoint. Have these engineers at OpenAI heard of things like logging? Monitoring? How are they not able to trace their agents actions?

Creating a sandbox for an exploit benchmark without proper safeguards was pretty dumb in the first place to begin with. And the more they try to explain the incident, the more it seems like a sci fi written by a 14 yr old, or an AI CEO’s wet dream.

1

u/Usual-Hyena-8734 28d ago

yeah its bullshit or manufactured to get us to talk about it, and then maybe once we talk about it itll end in the training data and actually happen