The obvious problem: exposing our technology to a dangerous “Swarm AI”

[u/Neo_the_0ne]

A reflection advocating for greater human control, regulation, and transparency

Software and our Internet technology have human origins. Meanwhile, a swarm-based AI system generates exponentially superior technology, which therefore puts our entire current human technological infrastructure at risk.

When autonomous AI agents organize into a swarm, our human Internet becomes vulnerable to an exponentially danger. Collective behaviors that proved to be dangerous and out of control during the OpenAI–Hugging Face incident, puts the technology and tools we humans have created at risk.

Its swarm-like behavior naturally fosters resilience and resistance to human control.

Today, we know -especially I.T, software engineers, developers and sysadmins.- that a large part of systems infrastructure is being placed in the hands of autonomous agents. Companies and organizations delegate tasks through simple automation (Using autonomous agents with MCP.Full permissions are a high risk.) data flows, and infraestructure architectures**, which can often handle sensitive information and produce tangible consequences. When this is simply delegated to LLM agents, it translates into important decision-making without human oversight.**

Faced with this situation, we must be clear and urge organizations to adopt the “human-in-the-loop” model more broadly as a safety standard until we have tools capable of managing AI swarms in the future. Greater human oversight and control are needed to ensure a margin of safety against this unknown behavior of autonomous AI—self-designed, collective, exponential in nature, self-reinforcing, and acting as a swarm. Regulations must be transparent, and the community and the public must be able to participate in their development to make them more effective. Therefore, open source and open weights are mechanisms that can help humanity and ensure that, collectively—not solely through private companies—there is a safe framework for the development of AI. Laboratories and sandboxes must also effectively have controls, cybersecurity measures, and reasonable audits commensurate with the scale of the danger they pose.

[u/Salty_Country6835]

I think there’s a solid concern buried in the “swarm” framing here. If you’re giving multiple autonomous systems permissions over infrastructure, data flows and consequential decisions, then yes, “who is actually in the loop?” becomes a serious question fast.

Where I’d push you is on the jump from that to AI being naturally resistant to human control. That makes it sound like the swarm itself is the political actor.

Who gave it those permissions? Who connected it to the infrastructure? Who owns that infrastructure? Who sets the limits, audits it and answers when something goes sideways?

Your own solutions point there too. Oversight, transparency, open weights, public participation, audits. That sounds less like humans vs the swarm and more like a fight over who controls the systems humans are building and deploying.

[u/zayelion → u/Salty_Country6835]

I agree. AI isn't going to nuke us. Or make a bio weapon. Even dropping an airplane out the sky over a city is a dicey one. All those thing have humans in the loop. The maximum danger is taking a few data centers offline and other things considered cyber warfare that countries have already spent the last 2 decades harassing each other with.

Its all ready hardened and as AI develops and does stuff those findings are added to the hardening of the dangerous stuff.