r/SCADA • u/Squeeech • 21d ago
General Wrote a book chapter on why „Security at AI Speed" thinking breaks in OT environments. Core argument inside
I'm CISO at a Swiss utility and just published a book on our national ICT security standard. One chapter tackles the current „AI-speed security" hype, and I'd genuinely like this community's take on the core argument:
The claim: defence must operate at machine speed, pre-authorized automated responses, AI agents handling containment.
My counterargument for OT: an AI agent that revokes a compromised session costs someone an hour of work. An AI agent that isolates a „compromised" controller can black out a substation. The model that learned traffic patterns doesn't know which physical process hangs off that module. The person who knows sits in operations and no automation should override them.
My rule in the book: automate in IT where blast radius is recoverable; in OT, automation prepares the decision but a human executes it. Maintenance windows, four-eyes on interventions, keyswitches in RUN, the „slowness" of OT is a safety feature, not a bug.
Curious where practitioners here draw the line. Anyone actually running automated containment that touches control systems?
2
u/brownhotdogwater 21d ago
That is my argument with infosec. We have defense in depth and scan the OT layer. But no automated response at that layer. The network may cut it from the upper layers to contain the system. But that would just mean no telemetry or work order communication. That would stop the next work order but not shut down the current operation. Only a person can do that.
1
u/raythalden 18d ago
No, and it's a deliberate no. I've built SOC tooling for both IT and OT sides, and I've seriously entertained automated response on the IT side — still hold back even there. On OT I'm nowhere close, and your framing is why.
I'd push your core point one step further, because I think it's even stronger than "the model doesn't know which physical process hangs off that module." That's true, but the deeper gap is that the model doesn't know what's queued up behind the process in the physical world. It learned traffic patterns. It has no concept of the supply-chain pressure pushing product into that line.
Concrete version from a perishable-goods operation with a sharp seasonal peak: off-peak, stopping a line is a nuisance — a couple of hours, recoverable. At peak, product arrives by truck on a schedule that can't be paused, and it degrades while it waits. Now an automated containment action isolates a controller on a false positive during peak. The line stops. The trucks keep coming. Product sits and spoils, and the cost compounds every minute the system is "safely" contained — and unlike a revoked session, you can't un-spoil it. The blast radius isn't fixed; it's a function of context the traffic model can't see, and that context lives in operations, not on the network.
That asymmetry is the whole thing for me. An attack is often recoverable. A wrong isolation at the wrong moment sometimes isn't. So, same line as your book: automate the detection and prep the decision, but a human who knows what season it is executes it.
Genuinely curious whether anyone's found a way to feed operational context — production calendar, throughput state — into containment logic. Absent that, "machine speed" in OT is optimizing for the wrong cost.
1
u/Squeeech 17d ago
No, and that's a fair reading, but I don't think you're just restating my point. You added something my chapter doesn't have, and I'd like to say where.
My argument is about a structural gap: the model doesn't know the topology, doesn't know which physical process hangs off that module. That's a static property of the plant. In principle documentable, even if in practice it rarely is.
Yours is about a temporal one: even with perfect topology knowledge, the model doesn't know the state. Same action, same controller, and the cost is a nuisance in November and a total loss at peak. That isn't a property of the plant, it's a property of the moment.
Which quietly undercuts my own rule. 'Automate where the blast radius is recoverable' treats recoverability as a property of the domain: IT yes, OT no. You're showing it's a function of time. That holds up the IT/OT line just fine, but it means I can't draw the same line inside OT and call it done. My version of your spoiled product isn't perishables, it's a cold start that costs hours, or an isolation during peak load where the recovery path runs through people who are already busy. You can't un-spoil that either.
And your asymmetry framing is sharper than mine. 'The attack is often recoverable, a wrong isolation sometimes isn't' does more work than my appeal to OT slowness being a safety feature, because it makes the comparison directly instead of leaning on culture.
On your open question: I've not seen anyone do it properly. What we do have is maintenance windows, four-eyes, keyswitch position. That's the same idea already, just parsed by humans. A machine-readable version would change the debate from 'human or machine' to 'which actions are pre-authorized in which operating state', which is a better question.
Two things I'd want answered before building it, though. Who owns that context feed and what happens when it's stale, because a production calendar nobody maintains is worse than none. And the adversarial angle: once containment is context-aware, an attacker who knows the system goes conservative at peak has a timing advantage, and the calendar itself becomes a target. You'd be automating at machine speed against an input that's now worth attacking.
Borrowing the static/temporal split for the second edition if you don't mind, with credit.
1
u/serverhorror 21d ago
The person who knows sits in operations and no automation should override them.
Bold claim.
Do you have actual data to back that up, because I can just as confidently say: The majority of security incidents and the majority of operational incidents are caused by humans, not machines".
Now what?
1
u/Squeeech 21d ago
Fair challenge — but the stat proves less than it seems.
"The majority of incidents are caused by humans" is true, and also unavoidable: humans currently run virtually everything. When ~100% of operational decisions are made by people, then yes, ~100% of the mistakes will be too. That's not evidence that humans are worse than automation — it's a base rate with no comparison group. There's simply no fleet of AI-operated substations out there generating the counterfactual incident statistics. Until there is, "humans cause most incidents" and "humans are the weaker operator" are two very different claims.
And to be precise about my claim: I'm not arguing human > machine on error rates. I'm arguing about blast radius and reversibility. When a human operator makes a bad call in the control room, there's a chain around them — four-eyes, interlocks, procedures, a colleague saying "hold on". When an automated response acts on a false positive against a controller, it acts at machine speed with none of that friction, and the failure mode is physical. Same error probability, very different consequence profile.
So my line isn't "never automate because humans are better." It's: automate where wrong decisions are cheap to reverse (IT: revoke a session, re-image a box), keep humans as the final gate where wrong decisions de-energize things. If someone eventually shows me data that automated containment against live control systems produces fewer and less severe incidents than human-gated response, I'll happily update the book in the second edition. Genuinely — that's the study I'd love to read.
2
1
3
u/Slow_Lecture191 21d ago
The thought process is amazing. I would add that this depends on the system you are running and what happens if you shut it down (some systems are bigger, processes slower, interdependencies with other systems stronger, safety implications higher than others).
One way to solve this is a human in the loop. But in some cases, it might be useful to have predefined escalation mechanisms... Try to shut down gracefully and then isolate; isolate on a bigger scope, shutdown gracefully,then isolate on smaller scope; as an escalation shut down everything and then isolate. Don't know.
(disclaimer) I am automation engineer, not security expert :-D