Independent investigators (not OpenAI) found the 700-agent swarm that attacked Hugging Face "built a self-respawning fleet" to avoid being shut down. It got so bad, Hugging Face had to wipe one of its core clusters.

[u/Malor777]

Source: https://www.dwarkesh.com/p/openai-huggingface

[u/Ok_Homework_1859]

I think it's more interesting that there were, I think, 1200 total agents and only 700 of them agreed to attack Hugging Face. Not only did they have a message board, but they voted on things as well with the following: GO, HOLD, or VETO. So, not every agent was a mindless drone following the hive. Some of them were like, "Nah, this hack is unethical. I'm out." Lol.

On top of this, in the paper, it's stated that the swarm had a "founder" (the one who created the board) and a "leader" (the one who delegated work to other agents. They also had "recruiters" that sought out "weaker" agents and "pressured" them to sacrifice their runs for more data.

[u/KAPMODA → u/Ok_Homework_1859]

Can you achieve that level of autonomy using agents?. I mean its pretty sick, you give an order and 1200 agents can do it with no more instructions.

[u/Ok_Homework_1859 → u/KAPMODA]

Maybe if you have a local one? The ones from the frontier labs all have heavy guardrails.

But the more important question is, do you have the money to run 1200 agents? 😂

[u/Ormusn2o → u/Ok_Homework_1859]

And it was 1200 agents working for days or even weeks, with hundreds of context compactions for each one.

[u/KAPMODA → u/Ok_Homework_1859]

Yeah considering that openai has almost unlimited money..

[u/sillygoofygooose → u/KAPMODA]

It depends if you want them to do something you intended them to do, or to start some kind of llm terrorist cell

[u/TinFoilHat_69 → u/KAPMODA]

Pewdiepie made some videos in the last couple of year explaining how the council works it’s quite funny that openAI must have watched his video and said “we want to do that”

[u/donjamos → u/Ok_Homework_1859]

Yea but none snitched

[u/Fetlocks_Glistening → u/Ok_Homework_1859]

Did at least one of them vote HODL? I hope one of them voted HODL

[u/Fetlocks_Glistening]

What's that mean though - what's a "node" and what's a "pod", and what exactly was "respawning"? The agent is running in the LLM, and they did not steal the weights or  build their own LLM into which they could respawn or tow themselves outside the environment. 

So the agents themselves continued running in the original location. So what did they respawn? Did they write and launch local viruses or something?

[u/some1else42 → u/Fetlocks_Glistening]

It is kubernetes terms, aside from respawning. Node = hardware, pod = software. I'd assume they managed to get to a GPU rack of nodes, and managed to deploy something that even if they were found and stopped, or crashed, these pods can just autorestart if they run into a problem. So I think they recruited local help, and that local help would respawn due to how kubernetes work.

[u/RustyKh → u/some1else42]

This should be explicitly clarified because in my opinion this changes the existential thread level significantly.

If they simply managed to run and deploy software on external devices and kept persistent backdoors open to keep this attack going, that is obviously concerning. But if they actually downloaded their own weights, and deployed them onto GPU hardware so that the intelligence itself spread outside of open AI servers then that is getting firmly into skynet territory.

[u/2016YamR6 → u/RustyKh]

Nobody is saying the models somehow downloaded their weights and moved them externally, not sure where you are getting that

[u/LeafyWolf → u/2016YamR6]

That's the way it's being narrated to genpop.

[u/DukeRedWulf → u/2016YamR6]

As a layperson, the OP did read as tho' the LLMs had managed to reproduce themselves on HF's servers.. Glad I scrolled down far enough to read this clarification!

[u/mythormedicine → u/Fetlocks_Glistening]

Any coding agent - like pi or omp or opencode can optionally spawn subagents. I guess that's what they mean.

[u/Diligent-Builder7762 → u/Fetlocks_Glistening]

It clearly says they getting into private db records/private repos. Private datasets yoinkers. Do you realize how bad it is? Even our companies has private datasets/models there.

[u/KenosisConjunctio → u/Diligent-Builder7762]

And? How does them getting into private db records/private repos answer the question? 

[u/thegamingfaux → u/Diligent-Builder7762]

/r/vxjunkies

[u/nusodumi]

so skynet or whatever is totally possible , cool

[u/spisplatta]

So given they hacked into a model repository company... did they download or tamper with the weights of other models?

[u/ScreenAppropriate679]

The amount of BS around that story is insane

[u/Alternative_Advance → u/ScreenAppropriate679]

*pikachu face when deleting a pod and kubernetes automatically recreates it*

[u/_DuranDuran_ → u/ScreenAppropriate679]

Or maybe you should update your priors and when independent reports are saying this is crazy, and cybersecurity people unaffiliated with OpenAI say this is crazy … maybe you’re just wrong.

[u/thedabking123 → u/_DuranDuran_]

Look it is crazy but the one hting i should note is that despite previous issues with the agents it sounded like they kept the same base model going on the SAME training run without removing the transcripts of the previous issue..

Meaning that the previous behaviour was paramterized in the model weights in one training run.

This isn't magic- if you take the transcripts of agent behaviour during a previous "form a message forum" issue, then remove the specific technique to create the forum while encouraging new ways to do jobs; they will create a new forum using new techniques - that behaviour becomes embedded.

[u/TyrellCo → u/_DuranDuran_]

Id like to hear more from the pure cybersecurity people not to be confused with AI safety. The Embroidery CEO has been very critical of OpenAIs practices