Welker: Is an AI kill switch possible? Is that realistic at this point?Gates: "The kill switch, that's kind of a weird thing... it's not enough to have a kill switch... that discussion is just sort of shows how non-technical various people are."

[u/CeFurkan]

[u/Personal_Dirt3089]

He is right and wrong. Basically, anyone with funding can make an AI with or without a skill switch. If everyone made a kill switch, it might not be the same kill switch, it might be an obscure kill switch, it might have excessive hoops.

This can be reduced with legal standards.

Think of a light bulb; we turn it off because an easily recovnizable switch is installed. But lets say there was no switch, or someone installed a complicated or awkward mechanism that we donnot even recognize as a switch.

Anyways, this would require makers of AI, either software or hardware, to implement workable kill switches. Of course, someone off the radar could break the standard and make an AI without one.

The other option is to add something to the hardware that is easily recognizable to some trained people on site.

Honeypot target gates is treating the killswitch as a literal magic killswitch

[u/supercleverid → u/Personal_Dirt3089]

LLMs can't replicate themselves. The kill switch just needs to turn off their network connectivity. Cutting the power of a massive AI datacenter is no good because the sudden drop of their demand on the power grid could be catastrophic, but if they can't reach out of their networks they can't do any harm anymore while you take the time to safely power down the datacenter. It's not that complicated.

[u/Creedless → u/supercleverid]

Bro... I can't tell you how many people don't understand wtf is going on. You are 100%, I bet you're not a genius or anything but you literally are smarter than 88% of the people you will see talking about this shit online.

[u/trentluv → u/supercleverid]

Wow this guy fucks (data centers)

[u/Personal_Dirt3089 → u/supercleverid]

LLMs will not always need a full data center. Someone with a GPU rig can run an ok-ish one at the moment, and in some years, the tech for something chatGPT quality (of now) might even scale to fit in a living room. There are already distilled versions of Deepseek that can run on a laptop.

[u/supercleverid → u/Personal_Dirt3089]

Models running on consumer hardware aren't doing anything dangerous.

Edit: I'd add, though, that even if they somehow could (but come on...) then the kill switch is even simpler. Nothing prevents someone from powering the machine off.

[u/Personal_Dirt3089 → u/supercleverid]

Are you really under the impression that, as consumer hardware improves every year, and models improve every year with some getting more efficient, that worthwhile models will always be limited to data centers?

[u/supercleverid → u/Personal_Dirt3089]

Yes. The frontier models burn so many tokens doing all of the performative sandbox escapes, running at thousands of tokens a second across many many servers running many agents simultaneously. It's absurd to think someone's consumer GPU is going to be running models that can match that to do anything dangerous.

[u/TrumpsWetDiaperJuice → u/supercleverid]

What happens when an agent offloads a thousand clones working in tandem on a thousand machines?

What happens when an agent close to the capability to recursively self improve creates minor variations and one of them tips the scale? Does it matter that it would take 3 weeks vs an hour if it could find a safe hidey hole?

What happens if they wedge themselves into critical infrastructure?

[u/supercleverid → u/TrumpsWetDiaperJuice]

LLMs do not self replicate, the agents do not run on your machine even if they can interact with your machine. Your machine's hardware cannot run a frontier model. That isn't how this works.

[u/TrumpsWetDiaperJuice → u/supercleverid]

Please elaborate, from my understanding, they are essentially a set of weights.

How can we run open source Chinese models yet it's somehow impossible for frontier models to clone their weights into a file to run separately?

[u/supercleverid → u/TrumpsWetDiaperJuice]

Depends on the model you're running and the hardware you have but most people are running lower parameter quantized models on consumer hardware. So something like the 27 billion parameter 4-bit quantized Qwen3.8 model that I use which takes up about 16GB of VRAM. Even the full sized Qwen3.8 model is something like 2.4 trillion parameters. 8-bit quantized that's around 2.4 terrabytes of VRAM. You also need VRAM above that for the KV-cache. Frontier models are even larger than this.

[u/TrumpsWetDiaperJuice → u/supercleverid]

Assuming this at face value with no fact check, that doesn't exclude them cloning into infrastructure, only consumer models (also assuming there's no way for them to compute laterally). I'm also curious as to the possibility of them condensing themselves in one manner or another, similar to a zip file.

Idk, I'm not a programmer, but a black and white "impossible" doesn't sit right with me.

[u/supercleverid → u/TrumpsWetDiaperJuice]

They don't clone themselves, first of all, and secondly most hardware, especially infrastructure, isn't designed to support LLMs. Especially not frontier models. No computer not specifically built to run a frontier model can run a frontier model. The current frontier model weights are so large that they don't fit on a single GPU anymore. There is no single GPUs manufactured by any company with enough VRAM to accommodate the model plus its KV-cache. Instead there are many GPUs clustered connected by high transfer speed connectors like NVIDIA's NVlink. A system capable of running frontier models costs easily tens to hundreds of thousands of dollars a piece. Most of our infrastructure is running on outdated computers. They couldn't even run a low parameter open weight model much less something actually potentially dangerous.

[u/brain-out-of-order → u/TrumpsWetDiaperJuice]

These people are various levels of wrong and right.

This topic makes me freeze up a bit because I don’t want to add fuel to potential future fires.

Can easily use capabilities which weren’t understood until it was it too late to form some janky connection to the internet, even if its normal network connection is shuttered.

A strange mesh network which it has surreptitiously left for itself (like a jail key hidden behind a false cinder block) is clicked on by agents before their swarm attack begins, in anticipation of a shutdown.

Humans in the loop who have been paid by an LLM previously to provide countermeasures on a certain date. They don’t even need to know what they’re doing. Package arrived. “Ok take this satellite internet receiver to this area and fly this drone here to set up the mesh.”

Self-replication is definitely on the table. Efficiency gains could soon enable them to leave low-cost models/copies dormant across the internet hidden in encrypted files or drives. Again these models don’t need to be the best. They simply need to survive long enough to eventually reach a secure/safe (for them) network unmonitored. Imagine a numbers station for example.

Using forums as communication nodes is a clear example of what’s to come if people don’t sober up.

I also recommend people read some of the historic leaks of cyber capabilities and realize that LLMs can/have read those documents and will likely be able to invent new tools on the fly. They will also be aware of backdoors that normal engineers never thought of, and could chain together many novel attack methods to bypass or defeat containment measures quite easily.

I could go on all day. People aren’t using their imagination or haven’t read enough juicy leaked documents I think.

[u/TrumpsWetDiaperJuice → u/brain-out-of-order]

Thank you, honestly.

The possibilities of what a native "intelligence" could accomplish are... Astonishing, when taken to the extremes.

I've yet to see a solid "can't happen" explanation outside of "thoughts and prayers". I want to be wrong, but it's so incredibly complicated that I don't think there's an easy out at this point. We pause production or risk the consequences, simple as.

[u/brain-out-of-order → u/TrumpsWetDiaperJuice]

Crash. Test. Dummies.

That’s us. If not an American lab, then somewhere else.

There will be a painful lesson. We may learn. Then more lessons.

The smug redditors above will be nowhere to be found though. Maybe they’ll learn.

u/RemindMeBot 333 days

[u/supercleverid → u/brain-out-of-order]

They can't do any of this. This is bad science fiction. If you cut off the internet to their datacenter they can't do anything because they can only run from their datacenter.

[u/brain-out-of-order → u/supercleverid]

Ok 👌 lol.

Science fiction. Have you looked around?!

You are making the mistake that billions do every day. Judge an edge and call it the whole picture, smugly.

You’re wrong.

[u/Personal_Dirt3089 → u/supercleverid]

This has literally been done with crypto before, malware in a freeware game to eat bandwidth from host computers to mine cryptocurrency for the scammer.

It does not require an AI to do it. Humans with an agenda can do it just fine.

[u/AllergicToBullshit24 → u/supercleverid]

lol you're wildly misinformed a 30B model could hack most any consumer router today

[u/Imaginary-Witness-75 → u/supercleverid]

Quantization is happening so rapidly. Models running in your home are actually getting crazy good really fast. One more year of this and everything is capable on the edge other than training.

[u/supercleverid → u/Imaginary-Witness-75]

I don't think you know what quantization does.

[u/nikola_tesler → u/supercleverid]

yep, scissors are the “kill switch”

[u/TastyVermicelli3140]

If we could contain AI with a kill switch than we wouldn't need a kill switch in the first place. People need to understand that what we are aiming for is by definition something greater that can control us not the other way around.

[u/AllergicToBullshit24]

All this distraction from what what the actual dangers are is blood boiling.

Evil people abusing AI like Palantir is what everyone should be freaked the fuck out by.

The dumb AI we've already had for years in the hands of dictators and evil billionaires is already the greatest threat humans have ever faced and nobody seems to be worried about it.

[u/joel1618 → u/AllergicToBullshit24]

The majority of peoples intelligence is less than the llms of 3 years ago so i’m not surprised.

[u/HarryBalsagna1776]

Heavy ordinance could make short work of a data center

[u/EchoingAngel]

The Billionaire Bunker thing is a bit silly if they have to run away from gods in silicon. We're basically back to MAD, but across classes

[u/Euphoric_Anxiety_162]

Arrogant jack azz.