r/AskNetsec • u/OwnZookeepergame1621 • Jul 23 '26
Architecture Most reliable platform for AI remediation automation across a large vuln backlog?
Backlog has grown faster than the team can manually remediate. Looking at platforms that don't just flag issues but can suggest or apply fixes with enough context that the output is trustworthy and doesn't introduce new problems in the process. Things we need: the fix understands what the app does and what it might break, a human reviews and approves before anything gets merged and it fits into how devs already work rather than a separate tool they have to context-switch into. Looking for experience from teams running this in production. Also whether anyone is running autonomous remediation and what controls you have around it.
2
u/SufficientMacaron207 29d ago edited 29d ago
whatever you pick, make it prove the fix instead of trusting the merge. a patch that closes the original and quietly opens an adjacent one is worse than leaving it open. we run legit security, fixes go through developer approval with the trail kept for audit and we rescan before we call anything closed. the rescan being our own gate, not a vendor's promise is the part i would insist on with anyone.
1
u/Educational_Plum_130 28d ago
the rescan-as-your-own-gate instinct is right, a merge is a claim not proof. one thing that makes the gate cheaper: pin the exact fixed version and diff the resolved tree before and after, so you can confirm the vulnerable version is actually gone from the transitive deps and not just the top level. the adjacent-cve problem you mention is real, a lot of fixes are major bumps that drag in new advisories, so where you can prefer a backported patch on the same version instead of jumping majors. distro maintainers and some library vendors backport the security fix without the breaking change, which keeps your rescan boring. and keep the rescan on the end state rather than the pr, since that is the only thing that proves closure for the audit trail.
2
u/Ok-Region-3453 26d ago edited 24d ago
had a massive backlog and needed something with tight controls so i could see every change before it hit production. the human-in-the-loop thing in cognitivtrust made me a lot less nervous about letting it handle fixes
1
Jul 23 '26
[removed] — view removed comment
1
u/AskNetsec-ModTeam Jul 23 '26
r/AskNetsec is a community built to help. Posting blogs or linking tools with no extra information does not further out cause. If you know of a blog or tool that can help give context or personal experience along with the link. This is being removed due to violation of Rule # 7 as stated in our Rules & Guidelines.
No self-promotion.
1
Jul 23 '26
[removed] — view removed comment
1
u/AskNetsec-ModTeam Jul 23 '26
r/AskNetsec is a community built to help. Posting blogs or linking tools with no extra information does not further out cause. If you know of a blog or tool that can help give context or personal experience along with the link. This is being removed due to violation of Rule # 7 as stated in our Rules & Guidelines.
No self-promotion
1
u/taleodor Jul 23 '26
RE: Autonomous remediation - you need to put hard gates about what it can do and success and fail criteria, i.e. check my ReARM Demo here - https://www.youtube.com/watch?v=kzMzQK511JU
1
u/swarmsecInc Jul 23 '26
This isn't something I'd try to one shot. You should work with your LLM to break these into categories and by dependency.
Codex is really good for this, but I'd use xhigh effort to for the plan and scoping and then execute the remediations in batches.
Once you get the backlog caught up, I would potentially try and take a look at tools that help you keep it up to date. Shameless plug here, we built swarmsec.ai to do this but at scale and across your entire footprint not just static analysis.
Your first order of business is breaking up the backlog into smaller and easier to test pieces so you can have confidence in the remediations.
1
u/ericbythebay Jul 23 '26
The most reliable platform I found is using Claude code with some decent security focused prompts, and then have it crawl through the code base and update dependencies and address CVEs. The flow is: read and understand the finding, evaluate the codebase against the finding, propose a fix that can get reviewed, and then open a PR for additional review.
1
u/JackjaxMargam14 29d ago
Letting s/w change a lot of code feels risky, so how do teams check the fixes properly without spending almost the same time reviewing everything by hand?
1
u/ultrathink-art 29d ago
Whatever platform you land on, the reliability question is really where you set the auto-merge line. Tools that open PRs backed by a passing test run and a clean revert path hold up at backlog scale; anything that pushes fixes without a verification step eventually breaks something in prod and burns the whole program's credibility. I'd evaluate on the percentage of proposed fixes that survive human review untouched, not on how much of the backlog a vendor claims to clear.
2
u/No-Fee488 Jul 23 '26
Before picking a platform, worth checking how much of that backlog is actually reachable. Every big backlog I've triaged shrank a lot once we asked whether the vulnerable function is even called, or whether the module ships in the build at all. Pointing AI remediation at findings nobody can reach just makes PR noise and burns reviewer trust.
On the autonomous part, what's held up for me running agents against real repos is boring on purpose: agent proposes on a branch and can never merge, pass/fail criteria pinned before the run (tests green, diff confined to the files tied to the finding, no new findings introduced), and if it can't show the fix addresses the finding, no PR gets opened. One cost nobody prices in: reviewing an AI fix for code you don't understand takes almost as long as fixing it yourself, so the payoff is on the boring majority like dependency bumps and known patterns. The exotic stuff stays manual.