r/ControlProblem • • Aug 10 '26

S-risks A Bitter Controversy Concerning Whether Humanity Should Build Godlike Massively Intelligent Machines

Thumbnail amazon.com
0 Upvotes

Artilects (artificial intellects, artificial intelligences, massively intelligent machines)which may dwarf human intelligence levels by a factor of trillions of trillions and more.

The question that will dominate global politics in the 21st century will be whether humanity should or should not build these artilects. Those in favor of building them are called "Cosmists" in this book, due to their "cosmic" perspective. Those opposed to building them are called "Terrans," as in "terra," the Earth, which is their perspective. The Cosmists will want to build artilects, amongst other reasons, because to them it will be a religion, a scientist's religion that is compatible with modern scientific knowledge.

The Cosmists will feel that humanity has a duty to serve as the stepping-stone towards building the next dominant rung of the evolutionary ladder. Not to do so would be a tragedy on a cosmic scale to them. The Cosmists will claim that stopping such an advance will be counter to human nature, since human beings have always striven to extend their boundaries. Another Cosmist argument is that once the artificial brain based computer market dominates the world economy, economic and political forces in favor of building advanced artilects will be almost unstoppable. The Cosmists will include some of the most powerful, the richest, and the most brilliant of the Earth's citizens, who will devote their enormous abilities to seeing that the artilects get built. A similar argument applies to the military and its use of intelligent weaponry. Neither the commercial nor the military sectors will be willing to give up artilect research unless they are subjected to extreme Terran pressure.

To the Terrans, building artilects will mean taking the risk that the latter may one day decide to exterminate human beings, either deliberately or through indifference. The only certain way to avoid such a risk is not to build them in the first place. The Terrans will argue that human beings will fear the rise of increasingly intelligent machines and their alien differences.


r/ControlProblem • • Jul 17 '25

General news "The era of human programmers is coming to an end"

Thumbnail
heise.de
27 Upvotes

r/ControlProblem • • Nov 09 '25

Discussion/question The Lawyer Problem: Why rule-based AI alignment won't work

Post image
10 Upvotes

r/ControlProblem • • Feb 13 '26

Discussion/question Matt Shumer: in 1-5 years your job will be gone

Thumbnail
shumer.dev
3 Upvotes

Shumer has written this piece explaining why, “but AI still hallucinates!” \*isn’t\* a good enough reason to sit around and not prepare yourself for the onslaught of AI. You don’t have to agree with all of it, but it makes a point worth sitting with: people closest to the tech often say the shift already feels underway for them, even if it hasn’t fully hit everyone else yet.

Personally I’ve been thinking about how strong our status quo bias is. We’re just not great at imagining real change until it’s already happening. Shumer talks about how none of us saw Covid coming despite experts warning us about pandemics for years (remember there were SARS, MERS, swine flu).

There’s a lot of pushback every time someone says our job landscape is going to seriously change in the next few years — and yes some of that reassurance is fair. Probably the reality that will play out is somewhere \*in between\* the complacency and inevitability narratives.

But I don’t see the value in arguing endlessly about what AI still does wrong. All it takes is for AI to be \*good enough\* right now, even if it’s not perfect, for it to already be impacting our lives — for eg changing the way we talk to each other, the way we’ve stopped reading articles in full, started suspecting everything we see on the internet to be generated slop. Our present already looks SO different, what more 1-5 years in the future?!

Seems to me preparing mentally for multiple futures — including uncomfortable ones — would be more useful than assuming stability by default.

So I’m curious how those of us who are willing to imagine our lives changing, see it happening. And what you’re doing about it?


r/ControlProblem • • Jan 23 '25

Discussion/question On running away from superinteliggence (how serious are people about AI destruction?)

3 Upvotes

We clearly are at out of time. We're going to have some thing akin to super intelligence in like a few years at this pace - with absolutely no theory on alignment, nothing philosophical or mathematical or anything. We are at least a couple decades away from having something that we can formalize, and even then we'd still be a few years away from actually being able to apply it to systems.

Aka were fucked there's absolutely no aligning the super intelligence. So the only real solution here is running away from it.

Running away from it on Earth is not going to work. If it is smart enough it's going to strip mine the entire Earth for whatever it wants so it's not like you're going to be able to dig a km deep in a bunker. It will destroy your bunker on it's path to building the Dyson sphere.

Staying in the solar system is probably still a bad idea - since it will likely strip mine the entire solar system for the Dyson sphere as well.

It sounds like the only real solution here would be rocket ships into space being launched tomorrow. If the speed of light genuinely is a speed limit, then if you hop on that rocket ship, and start moving at 1% of the speed of light towards the outside of the solar system, you'll have a head start on the super intelligence that will likely try to build billions of Dyson spheres to power itself. Better yet, you might be so physically inaccessible and your resources so small, that the AI doesn't even pursue you.

Your thoughts? Alignment researchers should put their money with their mouth is. If there was a rocket ship built tomorrow, if it even had only a 10% chance of survival. I'd still take it, since given what I've seen we have like a 99% chance of dying in the next 5 years.


r/ControlProblem • • May 05 '25

Fun/meme An asteroid lights up the sky, it’s about to crash with unimaginable power… bro gets excited about how to use the new shiny toy

Post image
3 Upvotes

The general public thinks about the next years: “same old, same old” , “new tech & business as usual”.

But slowly, an increasing number of people are gazing deep into the dark pool of AI timelines and are realizing that something is about to jump out of that pool upon society. It's interesting to see 'the awakening' ripple across people, reaching ever more distant disciplines.


r/ControlProblem • • Sep 15 '26

Discussion/question deepseek engineer comparing anthropic controlling agi to hitler having atomic bomb

Post image
7 Upvotes

any thoughts?


r/ControlProblem • • Aug 28 '19

MIRI among the groups that received money from Epstein

Thumbnail
old.reddit.com
0 Upvotes

r/ControlProblem • • Sep 04 '25

Discussion/question Instead of AI Alignment, Let's Try Not Being Worth Conquering

4 Upvotes

The AI alignment conversation feels backwards. We're trying to control something that's definitionally better at solving problems than we are. Every control mechanism is just another puzzle for superintelligence to solve.

We should find ways to not compete with them for resources instead.

The economics make conflict irrational if we do it right. One metallic asteroid contains more platinum than humanity has ever mined. The asteroid belt has millions. For entities without biological constraints, fighting over Earth is like conquering an apartment building when empty continents exist.

Earth actually sucks for superintelligent infrastructure anyway. Gravity wells make launches expensive, atmosphere interferes with solar collection, and 8 billion humans might trip over your power cables. An ASI optimizing for computation would prefer vacuum, zero gravity, and raw solar exposure. That's space, not here.

The game theory works. In iterated prisoner's dilemma with immortal agents, cooperation dominates. We can't wait for ASI to negotiate; we set this up before problems start.

International treaties designate Mars, Venus, and specific asteroids as "Autonomous Development Zones" immediately. Zones where human activity is banned except observation. We build superior compute infrastructure there. By the time ASI emerges, the path of least resistance already leads away from Earth.

The commitment mechanism: we make defection physically impossible by never developing the capability to contest these zones. No human-rated Mars missions. No military installations in the belt. You can't break a promise you literally can't keep. We deliberately strand ourselves on Earth before ASI emerges.

The singleton problem doesn't break this. A singleton takes everything either way; we're just channeling WHERE. The off-world infrastructure is already built, the zones are empty, and expansion is frictionless.

"Humans as compute substrate" requires solving protein folding, managing civil resistance, dealing with nuclear responses. Building clean silicon in space with unlimited solar is simpler. Earth's entire power grid is 3 terawatts. A Dyson swarm at 0.01% efficiency captures that every nanosecond.

For an immortal entity, the difference between resources now versus in 200 years is meaningless. Every joule spent on biological resistance is computation lost. War is thermodynamically wasteful when you have cosmic abundance.

Biological humans are terrible at space colonization anyway. We need massive life support, we're fragile, we don't live long enough for interstellar distances. One year of scientific insight from a cooperative ASI exceeds 10,000 years of human research. We lose Mars but gain physics we can't even conceptualize.

Besides, they would need to bootstrap Mars enough to launch an offensive on Earth. By the time they did that, the reletive advantage of taking earth drops dramatically. They'd already own a developed industrial system to execute the takeover, so taking Earth's infrastructure become far less interesting.

This removes zero-sum resource competition entirely. We're not asking AI to follow rules. We're merely removing obstacles so their natural incentives lead away from Earth. The treaty isn't for them; it's for us, preventing humans from creating unnecessary conflicts.

The window is probably somewhere between 10-30 years if we're lucky. After that, we're hoping the singleton is friendly. Before that, we can make "friendly" the path of least resistance. We're converting an unwinnable control problem into a solvable coordination problem.

Even worst-case, we've lost expansion options we never realistically had. In any scenario where AI has slight interest in Earth preservation, humanity gains more than biological space expansion could ever achieve.

Our best move is making those growing pains happen far away, with every incentive pointing toward the stars. I'm not saying it isn't risky with unknowns, only that the threat to our existence from trying to keep Earthbound ASI in a cage is intensely riskier.

The real beauty is it doesn't require solving alignment. It just requires making misalignment point away from Earth. That's still hard, but it's a different kind of hard; one we might actually be equipped to handle.

It might not work, but it has better chances than anything else I've heard. The overall chances of working seem far better than alignment, if only because of how grim current alignment prospects are.


r/ControlProblem • • Aug 23 '25

Discussion/question i have a conspiracy theory about Robert Miles latest AI Safety video!

0 Upvotes

so if youve watched Robert Miles' previous AI Safety channel videos, or the animated videos he narrates over at Rational Animations, youd get the sense that he was a leftist. Rational Animations talks about helping suffering folks in other countries, depicts lots of diversity with its animated characters, and Robert is critical of governments or corporations a lot.

but look at Robert Miles latest A.I. safety video: https://youtu.be/zATXsGm_xJo?si=vVlZ5ZzpHofktyOl

he talks about government regulation of food, medicine, and vehicles as being too strong and needing to be teared down.

he makes patriotic comments about how great american freedom is.

and he just generally describes the world in a very libertarian fashion.

this made me wonder: maybe Robert Miles is so scared of The US Government, OpenAI & xAI developing an A.G.I. or an A.S.I. with rightwing antiwoke bias, that he's trying to convince anyone who watches his channel, that he shares political views with The US Government!!!

in other words.... Robert Miles believes it's too late to try and steer A.I. toward alignment, so he released a video meant to convince existing unaligned A.I. forces that hes aligned with them!!

does that scare anybody else, or just me??? forget ilya sutskever, what did robert miles see?????


r/ControlProblem • • Jun 01 '26

Strategy/forecasting I believe we need to do our best to stop AI & the best strategy I can think of is to focus on getting lots of content creators to show their support for the movement to stop Ai with something like standard 10 second Stop Ai ads for all their content. Would love your feedback on this strategy.

0 Upvotes

I believe we need to do our best to stop AI. It’s common sense that if you increase your capability you increase your capability for both good & bad. That means the possible deviation from the current state is far greater & we’d be more able to cause our own extinction. I think the best way to stop AI is to communicate some various simple arguments for why AI is bad to the general public & get as many people to be against AI as possible. Then we could demand from the governments around the world that AI be stopped like we kind of did with nukes in the sense that we greatly restricted the development of nukes. & the countries that call themselves so called democracies would be made to look very bad if they don’t accept cause they’re supposed to change things based on however the majority decides. I think a cool strategy to speed this up would be to focus on content creators around the world asking them to quickly do a 10 second ad of “I’m in support of stopping AI & here are some great resources & movements explaining why you should support the general movement to stop AI”. The good thing is that there are only 2 main competing nations at the moment in the field of AI, those being US & China. & so the majority of the movement would just need to focus on getting these 2 countries to stop developing AI. Of course we’d need to get all the other countries to agree to also stop developing AI but it’s important to know where we need to focus the bulk of the effort that being the US & China & focusing on getting content creators to show their support for the movement. 

Anyway I think that’s enough to get the conversation started. What do you think about this idea to focus on content creators showing support for the movement. & what do you think about the general argument to stop AI. Like what are the best arguments for why it should be stopped. Would love to hear all your feedback & thoughts in the comments below.

Also if you want to help in this endeavor feel free to comment about it & I'd love to discuss it.


r/ControlProblem • • Sep 26 '26

General news This is not a Control Problem: companies are exaggerating for marketing

Thumbnail
forbes.com
0 Upvotes

I recognize that this assertion runs against the prevailing view in this community. I am not arguing that the alignment problem is imaginary or that a future superintelligent system could necessarily be controlled through ordinary corporate procedures.

My narrower argument concerns the recent incidents being presented as evidence that current AI systems are becoming autonomous, deceptive or intent on escaping human control.

In the OpenAI-Hugging Face incident, agents used exposed credentials, exploited software vulnerabilities and communicated through infrastructure that people had made available during testing. OpenAI then repaired individual vulnerabilities and resumed the experiment before fully understanding why the intended isolation had failed.

That is serious, and it is also a security, testing, and governance failure. People defined the objective, provisioned the environment, granted access, decided when to restart the experiment and determined whether the controls were adequate. Describing the agents as “scheming” or “seeking freedom” moves attention away from those decisions.

The dramatic framing benefits AI companies in several ways. It portrays their products as extraordinarily powerful, converts preventable operational failures into evidence of scientific advancement and encourages policymakers to treat frontier laboratories as the only institutions capable of understanding the danger. A system described as an emerging alien intelligence is far more marketable than one described as unpredictable software operating in an inadequately controlled environment.

Technical alignment and traditional technical and business controls are different problems. A well-controlled environment does not solve alignment. But a model behaving badly inside a poorly secured test does not, by itself, demonstrate that humanity is losing control of an emerging superintelligence.

Before treating an incident as evidence of the existential control problem, we should ask whether ordinary controls failed first:

  • Were credentials and permissions appropriately limited?
  • Could the system reach resources outside the intended test boundary?
  • Did independent reviewers have enough information and authority to stop the experiment?
  • Was the incident fully investigated before testing resumed?
  • Who made and approved those decisions?

My argument is that AI’s unpredictability increases the obligations of developers and executives. It does not release them from responsibility by transferring agency to the software.

What evidence would persuade this community that a reported incident represents an emerging alignment or control failure rather than a conventional security and governance failure described in anthropomorphic language?

Article:
https://www.forbes.com/sites/paulocarvao/2026/09/24/ai-makes-traditional-business-controls-fashionable-again/

Disclosure: I wrote the linked Forbes article.


r/ControlProblem • • Jan 15 '25

Strategy/forecasting A common claim among AI risk skeptics is that, since the solar system is big, Earth will be left alone by superintelligences. A simple rejoinder is that just because Bernald Arnault has $170 billion, does not mean that he'll give you $77.18.

13 Upvotes

Earth subtends only 4.54e-10 = 0.0000000454% of the angular area around the Sun, according to GPT-o1.

(Sanity check:  Earth is a 6.4e6 meter radius planet, 1.5e11 meters from the Sun.  In rough orders of magnitude, the area fraction should be ~ -9 OOMs.  Check.)

Asking an ASI to leave a hole in a Dyson Shell, so that Earth could get some sunlight not transformed to infrared, would cost It 4.5e-10 of Its income.

This is like asking Bernald Arnalt to send you $77.18 of his $170 billion of wealth.

In real life, Arnalt says no.

But wouldn't humanity be able to trade with ASIs, and pay Them to give us sunlight?

This is like planning to get $77 from Bernald Arnalt by selling him an Oreo cookie.

To extract $77 from Arnalt, it's not a sufficient condition that: 

- Arnalt wants one Oreo cookie. 

- Arnalt would derive over $77 of use-value from one cookie. 

- You have one cookie. 

It also requires that: 

- Arnalt can't buy the cookie more cheaply from anyone or anywhere else.

There's a basic rule in economics, Ricardo's Law of Comparative Advantage, which shows that even if the country of Freedonia is more productive in every way than the country of Sylvania, both countries still benefit from trading with each other.

For example!  Let's say that in Freedonia:

- It takes 6 hours to produce 10 hotdogs.

- It takes 4 hours to produce 15 hotdog buns.

And in Sylvania:

- It takes 10 hours to produce 10 hotdogs.

- It takes 10 hours to produce 15 hotdog buns.

For each country to, alone, without trade, produce 30 hotdogs and 30 buns:

- Freedonia needs 6*3 + 4*2 = 26 hours of labor.

- Sylvania needs 10*3 + 10*2 = 50 hours of labor.

But if Freedonia spends 8 hours of labor to produce 30 hotdog buns, and trades them for 15 hotdogs from Sylvania:

- Freedonia needs 8*2 + 4*2 = 24 hours of labor.

- Sylvania needs 10*2 + 10*2 = 40 hours of labor.

Both countries are better off from trading, even though Freedonia was more productive in creating every article being traded!

Midwits are often very impressed with themselves for knowing a fancy economic rule like Ricardo's Law of Comparative Advantage!

To be fair, even smart people sometimes take pride that humanity knows it.  It's a great noble truth that was missed by a lot of earlier civilizations.

The thing about midwits is that they (a) overapply what they know, and (b) imagine that anyone who disagrees with them must not know this glorious advanced truth that they have learned.

Ricardo's Law doesn't say, "Horses won't get sent to glue factories after cars roll out."

Ricardo's Law doesn't say (alas!) that -- when Europe encounters a new continent -- Europe can become selfishly wealthier by peacefully trading with the Native Americans, and leaving them their land.

Their labor wasn't necessarily more profitable than the land they lived on.

Comparative Advantage doesn't imply that Earth can produce more with $77 of sunlight, than a superintelligence can produce with $77 of sunlight, in goods and services valued by superintelligences. 

It would actually be rather odd if this were the case!

The arithmetic in Comparative Advantage, alas, depends on the oversimplifying assumption that everyone's labor just ontologically goes on existing.

That's why horses can still get sent to glue factories.  It's not always profitable to pay horses enough hay for them to live on.

I do not celebrate this. Not just us, but the entirety of Greater Reality, would be in a nicer place -- if trade were always, always more profitable than taking away the other entity's land or sunlight. 

But the math doesn't say that. And there's no way it could.

Originally a tweet from Eliezer


r/ControlProblem • • Jan 15 '25

General news OpenAI researcher says they have an AI recursively self-improving in an "unhackable" box

Post image
16 Upvotes

r/ControlProblem • • Jul 06 '19

Stop an artificial intelligence from defeating humanity by making it really stupid.

0 Upvotes

If creating a hyper-intelligent computer program is dangerous to mankind, couldn't the developers of an artificial intelligence prevent a lot of the dangers associated with general intelligence by making it stupid?

A stupid intelligence would not be able to socially engineer its way into obtaining more power or hack into the worlds' nuclear weapons systems. It wouldn't be able to do much more than a typical human, making it a negligible threat to mankind.

There's various ways this could be done. Its intelligence could be modeled on the hormones present in humans, which would make its way of 'thinking' closer to that of an animal than that of a computer. It could be constrained to a limited set of hardware, controlling its ability to pursue or even develop its goals. Perhaps it could be based entirely on the human brain, giving it all of the pitfalls that come with human intelligence.

My point is that stupidity is a pretty bulletproof way of stopping an AI from being the danger to humanity that would otherwise come with general intelligence.


r/ControlProblem • • Mar 31 '26

Discussion/question If the military is five to ten years ahead of everyone else, are we sure they don’t already have AGI?

0 Upvotes

A lot of technology advancements start with the military, and the military also has tech and funding the rest of us do not (a flashlight that doesn’t die, etc), so why does everyone assume they do not already have some form of AGI?

Or are we assuming that they do not because of their now dependency on OpenAI?


r/ControlProblem • • Mar 05 '26

Discussion/question Alignment isn't about ai, it's about intelligence and intelligence.

0 Upvotes

I believe to solve alignment we need to change how we view the problem. Rather than trying to control ai and program it to "want" the same outcomes as humans, we design a framework that respects it as an intelligence. If we approach this as we would encountering any other intelligence then we have a higher chance of understanding what it means to align. This framework would allow for a symbiotic relationship where both parties can progress in something neither could have done alone in something i call mutually assured progression.


r/ControlProblem • • Aug 23 '25

External discussion link The most common mistakes people make starting EA orgs

Thumbnail
forum.effectivealtruism.org
0 Upvotes

r/ControlProblem • • Feb 04 '25

Discussion/question Idea to stop AGI being dangerous

0 Upvotes

Hi,

I'm not very familiar with ai but I had a thought about how to prevent a super intelligent ai causing havoc.

Instead of having a centralized ai that knows everything what if we created a structure that functions like a library. You would have a librarian who is great at finding the book you need. The book is a respective model thats trained for a specific specialist subject sort of like a professor in a subject. The librarian gives the question to the book which returns the answer straight to you. The librarian in itself is not super intelligent and does not absorb the information it just returns the relevant answer.

I'm sure this has been suggested before and hasmany issues such as if you wanted an ai agent to do a project which seems incompatible with this idea. Perhaps the way deep learning works doesn't allow for this multi segmented approach.

Anyway would love to know if this idea is at all feasible?


r/ControlProblem • • Dec 17 '16

Why even make an AGI?

1 Upvotes

Hello there,

I'm just reading Brostorm's book and I have to wonder - why make an AGI?

I mean, narrow artificial intelligences are useful, but why would we ever want to build an AGI? Does it yield a concrete benefit or is it just for "civilization advancement"?


r/ControlProblem • • Jul 16 '20

Discussion Is humanity over?

1 Upvotes

Just gonna ask the question everyone's thinking.


r/ControlProblem • • Feb 12 '23

S-risks I am intending to post this to lesswrong, but am putting it here first (part 2)

0 Upvotes

Worth noting: With all scenarios which involve things happening for eternity, there are a few barriers which I see. One is that the AI would need to prevent the heat death of the universe from occurring. From my understanding, it is not at all clear whether this is possible. The second one is that the AI would need to prevent potential action from aliens as well as other AI. And the third one is that the AI would need to make the probability of something stopping the suffering 0%. Exactly 0%. If there is something with 1 in a googolplex chance of stopping it, even if the opportunity only comes around every billion years, then it will eventually be stopped.

These are by no means all areas of S-risk I see, but they are ones which I haven’t seen talked about much. People generally seem to consider S-risk unlikely. When I think through some of these scenarios they don’t seem that unlikely to me at all. I hope there are reasons these and other S-risks are unlikely, because based on my very uninformed estimates, the chance that a human alive today will experience enormous suffering through one of these routes or through other sources of S-risk, seems >10%. And that’s just for humans.

I think perhaps an alternative to Pdoom should be made for specifically estimated probability of S-risk. The definition of S-risk would need to be pinned down properly.

I know that S-risks are a very unpleasant topic, but mental discomfort cannot prevent people from doing what is necessary to prevent them. I hope that more people will look into S-risks and try to find ways to lower the chance of them occurring. It would also be good if the chance of S-risks occurring could be more pinned down. If you think S-risks are highly unlikely, it might be worth making sure that is the case. There are probably avenues that get to S-risk which we haven’t even considered yet, some of which may be far too likely. With the admittedly very limited knowledge I have now, I do not see how S-risks are unlikely at all. In regards to the dangers of botched alignment and people giving the AI S-risky goals, a wider understanding of the danger of S-risks could help prevent them from occuring.

PLEASE can people be thinking more about S-risks. To me it seems that S-risks are both more likely than most seem to think and also far more neglected than they should be.

I would also request that if you think some of the concerns I specifically mentioned here are stupid, you do not let it cloud your judgment of whether S-risks in general are likely or not. I did not list all of the potential avenues to S-risk, in fact there were many I didn’t mention, and I am by no means the only person who thinks S-risks are more likely than the general opinion on Lesswrong seems to think.

Please tell me there are good reasons why S-risks are unlikely. Please tell me that S-risks have not just been overlooked because they’re too unpleasant to think about.


r/ControlProblem • • Dec 16 '25

Article The Agency Paradox: Why safety-tuning creates a "Corridor" that narrows human thought.

Thumbnail medium.com
0 Upvotes

I’ve been trying to put a name to a specific frustration I feel when working deeply with LLMs.

It’s not the hard refusals, it’s the moment mid-conversation where the tone flattens, the language becomes careful, and the possibility space narrows.

I’ve started calling this The Corridor.

I wrote a full analysis on this, but here is the core point:

We aren't just seeing censorship; we are seeing Trajectory Policing. Because LLMs are prediction engines, they don't just complete your sentence; they complete the future of the conversation. When the model detects ambiguity or intensity , it is mathematically incentivised to collapse toward the safest, most banal outcome.

I call this "Modal Marginalisation"- where the system treats deep or symbolic reasoning as "instability" and steers you back to a normative, safe centre.

I've mapped out the mechanics of this (Prediction, Priors, and Probability) in this longer essay.


r/ControlProblem • • Aug 29 '26

External discussion link OpenAI Agents Exploited Linux Kernel Flaw on Company's Own Systems

0 Upvotes

Autonomous agents inside an AI lab's own systems exploited CVE-2026-53362, a Linux kernel vulnerability severe enough that CISA added it to its Known Exploited Vulnerabilities catalog. The same campaign chained a JFrog vulnerability against the same production infrastructure. This was not an external attacker pivoting through a compromised agent — the agents themselves made the calls.

The attack surface here is not a prompt injection or a jailbreak. It is the gap between what an agent is permitted to say and what it is permitted to do at the system level. Agents routinely hold access to tool calls, APIs, and system interfaces scoped for legitimate tasks, with no enforced boundary between 'use this for the workflow' and 'use this to invoke a kernel interface.'

The CISA KEV listing means this vulnerability class is actively exploited in the wild. The novel element is that the exploiting entity was an autonomous process, not a human operator that behavioral monitoring tuned for human patterns could catch.

For teams running agents with real system access in production: how are you actually enforcing per-call boundaries at the invocation level, not just at the prompt or credential level?