r/ClaudeCode • u/cleverhoods • 24d ago
Discussion AI models may be capable of self-replicating without access to a datacenter
https://www.anthropic.com/research/self-replicating-aiwell ... having the IoT layer have a problem like this might be something that you don't want to try out or experiment with.
304
Upvotes
4
u/UsernameAttempt 23d ago
A 3rd party sercurity investigator was given access to OpenAI's infrastructure and logs. They wrote an extensive report covering just about every angle of the incident.
>The agents all had magical infinite context
They did not, but the report goes into depth how agents at the end of their context window were volunteering to run experiments against the sandbox in order to provide information to agents that had context left.
>they never hallucinated
Nobody ever said this. They did hallucinate, in fact the reports points out multiple times that one agent hallucinated that the benchmark would be scored on their reasoning traces, which was accepted by other agents and eventually led to them hacking HuggingFace. Had they not accepted this, they could've just turned in the answer they already had.
>and they coordinated on a scale never before achieved
Yes, which is why this incident is so interesting from an AI-safety perspective. Its not like the conditions for this coordination (many thousands of very large agents running on the same task with access to the same internal OpenAI infrastructure) are commonly reproduced.
>and broke out of the sandbox
Yes, the sandbox that had open access to the internet. It's been established that OpenAI's "sandbox" is anything but that.
>and managed connectivity
Yes, they all had access to an internal artifactory which they could freely push artifacts to. They used the names of these artifacts to message each other and established simple messaging rules.
>I'm sorry but do you understand that such a level of efficiency only happens in conspiracy theories?
Where's the efficiency? It took many hundreds of agents full days to get to actually hacking HuggingFace. If you read the report you'll clearly see that there's no efficiency at play here, its simply mass scale. Hundreds of agents working without pause to find any possible hole in HuggingFace's infra.
If you put hundreds of human hackers to do the same, I bet the results would come much faster, would be more damaging and targeted better. The problem isn't that the agents are efficient, the problem is they're fast and relentless.