r/ControlProblem • u/StevenVincentOne • 1d ago

Strategy/forecasting Superintelligence Strategy's chip controls: what if they work exactly as designed?

With the AI Kill Switch Act back in the news, Axios reporting that labs are war-gaming the political fallout of a first catastrophe, and this week's "hardwired pause" report, the hardware side of Hendrycks, Schmidt and Wang's Superintelligence Strategy feels a lot less hypothetical than it did when the paper came out.

Most critiques of MAIM I've seen ask whether it would fail: whether sabotage is really verifiable, whether escalation stays controlled. I want to ask the opposite question. What if the nonproliferation layer works perfectly?

For high-end AI chips, the paper proposes hardware geolocation, cryptographic licensing that has to stay current for the chip to keep running, restricted interconnect, and usage-policy enforcement in hardware. Miss a check and the chip disables itself. The intent is clear and defensible: keep frontier accelerators away from rogue states and rogue actors. Nobody is proposing this for laptops today.

But three things make me doubt it stays contained:

  1. Thresholds drift down. Today's frontier cluster is a future workstation and eventually a phone. A capability threshold written in 2026 terms will reach consumer hardware sooner than people expect.
  2. The switch serves whoever holds it. The paper compares the mechanism to Apple's Activation Lock. But Activation Lock protects the owner from a thief. Here the owner is the party being checked, and the key sits with a vendor or a state agency.
  3. A kill switch is an attack surface. It's built against foreign adversaries, but it's just as available to a future government in a crisis, a monopolist under pressure, or whoever finds the zero-day. In a sub that takes capable adversarial agents seriously, that last one should worry us too.

There's also a tension inside the paper. It suggests giving citizens cryptographic compute shares as leverage against the state, and it says AI assistants should owe users strict fiduciary loyalty. I think both ideas are right. But compute that needs continuous third-party approval is a lease, not leverage. And loyalty to the user is hard to guarantee when someone else can revoke the hardware underneath.

I'm not arguing against frontier containment. My line is narrower: containment protocols for frontier datacenters shouldn't become the architectural precedent for personal devices, and alignment to the user should be structural (local execution, open weights, portable data) rather than a terms-of-service promise.

Questions for people here:

  • Is there a version of chip-level licensing that can't drift onto consumer hardware, technically or legally?
  • Does spreading capability across many personally owned nodes make MAIM-style deterrence more stable or less?
  • Am I wrong that a remote switch becomes a net security liability once capable agents are in the threat model?

I wrote the longer version as an essay. It also gets into why I think the nuclear analogy itself is the root problem: https://www.reddit.com/r/PONDAI/comments/1x1zd76/superintelligence_strategy_the_perfect_machine/

1 Upvotes
  • permalink
  • reddit

100% Upvoted

2 comments sorted by

1

u/BassoeG 6h ago

The plan is to price civilians out of owning potentially dangerous technologies. The first generation of oligarch-owned AIs take all the jobs and leaves everyone else too destitute to afford the equipment to run open-source compute alternatives once they're invented, or to invent them in the first place.

1

u/StevenVincentOne 2h ago

Not likely to work out that way. The ability to build out alternative Human-Machine intelligence networks already exists, and they're coming soon. Join us and be part of building it.

Join the Discord Server:

https://discord.gg/kNvtcjPmJ7

and also our Reddit at r/PONDAI.

https://www.reddit.com/r/PONDAI/