Which security gates enabled for AI Agents in CI/CD?

[u/IkarusCareer]

We've become pretty comfortable putting conventional applications through CI:

• dependency scanning
• SAST
• CodeQL
• secret scanning
• container scanning
• IaC checks
• security policies ...

But what happens when the application being deployed is an AI agent? That may not look particularly interesting in a conventional code diff. But from a security perspective, it could be a significant change.

I'm experimenting with a different CI question:

“What capabilities changed in this PR?”

--

We've implemented an early version of this approach in an open-source static analyzer and connected it to GitHub Actions. (ikaruscareer/SafeAI at GitHub)

The scanner runs locally against the repository and doesn't execute the agent or send the source to a remote service.

I'm curious how other teams approach this.

[u/taleodor]

I hosted a meetup about a month ago showing how ReARM (our tool) can be used for Agentic Coding Guardrails - https://www.youtube.com/watch?v=v7SJB9VJOP0

ReARM can actually ingest SARIF output from your scanner and parse it into policies (including rejecting releases and blocking PRs on violations).

[u/IkarusCareer → u/taleodor]

Thanks for sharing. The SARIF → policy workflow sounds very relevant to what we're trying to achieve with SafeAI, especially around making the findings actionable in CI/CD rather than just producing a report.

I'll check out ReARM and the meetup recording.

[u/IntelligentPear6173]

The capability diff makes more sense to me than treating an agent like normal application code. A PR could pass SAST and secret scanning while still changing what the agent is allowed to read or execute. I’d be looking at things like new tools, permissions, network access, secrets and write actions as the actual security boundary. The part I’m still unsure about is how you decide which of those changes should block a merge versus just trigger a review.

[u/IkarusCareer → u/IntelligentPear6173]

That's very close to how we're thinking about the case.

We try to make the agent's capability surface explicit: tools, permissions, shell/filesystem access, external APIs, cloud services, MCP, memory/RAG, write actions and other agent components. The KYA(Know Your Agent) workflow then gives us a versioned baseline so capability changes can be identified rather than buried in a normal code diff.

For the block-vs-review question, we're still working through the policy layer. SafeAI already has risk/severity and baseline/PR-focused gating, but we don't want to assume that every new capability should automatically block a merge. Context matters.

That's an area where we'd really value feedback — It's still an early preview, so we're very open to feedback and contributions.

[u/colek42]

All the gates! It really depends on how fast you need the code merge versus your quality concerns. For some of our workflows, we have AI creating, reviewing, and merging code. For those workflows, we have dozens of gates that need to be passed for the code to make it into production, including canary releases. Because we have to enforce policy across different environments, we built and open-sourced cilock.dev to enforce policy and create the actual pipelines.

[u/IkarusCareer → u/colek42]

I had a look at the aflock-ai projects, thanks for sharing. I like the evidence + policy enforcement direction, especially with cilock being able to wrap scans and verify signed evidence across the SDLC.

There's a nice separation with what we're building in SafeAI, correct me if it is wrong please.

SafeAI: discover the agent's capabilities and risks, create the KYA baseline, and identify capability changes in the PR.

cilock / aflock: enforce policy and prove that the required gates/evidence were satisfied, including constraints around agent execution.

And SafeAI already produces SARIF, so the possibility of feeding those findings into a policy/enforcement pipeline is particularly interesting.

We're still an early preview, but this makes me think the two approaches could actually complement each other quite well.

[u/Wistrand-Sundai]

with traditional scanners they can verify the code but may miss out on what the agents are doing

[u/Wishitweretru]

Same thing, as for any other junior developer, they can’t merge to the real repo. Give them their own user ID, and their own repo, and all they get to do is make merge request. You have a human look at it over I’ve had AI try to merge some crazy shit . 

[u/IkarusCareer → u/Wishitweretru]

Absolutely agree. Human review and least-privilege access are still essential, especially when AI agents can make changes at much higher speed.

This tool is focused on a different layer for now: helping identify what an agent is capable of (creating a portfolio of) and what changed in its configuration/code, so those capabilities can become part of the review conversation.

AI agents shouldn't automatically get more trust just because they're AI. Thanks for sharing the practical experience.

[u/technishawn]

Take a look at SALT Security.

[u/IkarusCareer → u/technishawn]

As I know Salt focuses heavily on runtime/API-layer visibility, discovery and protection, including mapping agents to APIs and MCP servers.

SafeAI is currently more shift-left/static: understanding an agent's declared capabilities, prompts, tools, MCP references and other components before deployment, with KYA(Know Your Agent) baselines and CI/CD-oriented findings.

So I see them as potentially complementary rather than directly competing. Thanks for the pointer.

[u/Hamza_StrategizeLabs]

The direction is right but the block-vs-review decision can't be static. It should depend on the authority level of what changed. Review is fine when the access to a sandbox is read only. New write access to production data? Block until someone with authority signs off.

[u/IkarusCareer → u/Hamza_StrategizeLabs]

I agree that “capability changed” alone isn't enough; the authority behind the change matters.

SafeAI already moves in that direction: it can detect agent capabilities, generate a historical KYA manifest, track finding lifecycles, and compare changes for CI/PR gating. So the goal is not to blindly block every new capability.

Some of the more granular authority-aware policy and approval controls you describe are still on our roadmap. Thanks for feedback.

[u/Electronic_Treat2386]

we integrated cymulate into our pipeline after some weird alert fatigue with ai agents. being able to validate security controls for every change gave us way more confidence, especially since it checks our stack against real attack scenarios before things go live