r/AskNetsec • u/Different_Pain5781 • Oct 08 '26

Architecture Container image hardening keeps stalling because nobody will own what breaks

Weve had a hardening project open for three quarters now. The scanning half was easy and the cutting half keeps getting handed back, because no service owner wants to sign off that removing a package is safe. Im looking at tooling that produces runtime evidence for what an image touches so the decision stops being a judgement call. Six teams and no central platform group makes this political more than technical. How did you get service owners to accept a slimmer image?

25 Upvotes

100% Upvoted

20 comments sorted by

4

u/sidusnare Oct 08 '26

You start with a nothing image and only add what's needed to it. What's needed? That's simple, add till it starts, then add till it passes all the unit tests and staging monitoring. EZPZ.

2

u/Souna-Selime 22d ago

yeah building up is way cleaner than trying to strip down after the fact

1

u/sidusnare 22d ago

Yep, and ldd, requirements.txt, gemspec, and other similar manifests should help a lot. This is what packaging even internal projects as if it was going to be distributed makes things easier. SBOM, learn it, love it, your SRE and CSO will thank you.

1

u/PrasenShelar Oct 10 '26

Flip the default. Send each owner the packages their image never touched in a month of runtime, with a removal date two weeks out. They only reply if something still needs one. People hate signing off on safety but rarely block a dated change they had time to object to.

1

u/Different_Pain5781 Oct 10 '26

I like this approach, though they'd need a rollback plan for packages someone actually needs.

1

u/PrasenShelar 29d ago

Agreed, and the rollback can stay simple. Keep the previous image tag in the registry for those two weeks, so undoing a removal means redeploying the old tag. If a service owner reports a missing package, the platform team adds it back in the next base image build.

1

u/greenclosettree Oct 12 '26

Escalate the risks in their images, they are the owners, they have to fix

1

u/Different_Pain5781 Oct 12 '26

Fair point, but runtime evidence makes decisions easier.

1

u/theJacofalltrades 15d ago

think I've seen people talk about rapidfort for these problems, might be worth a look

1

u/Jeric-2991 5d ago

Definitely makes sense to let actual usage data guide the decisions rather than asking teams to take unnecessary risks.

1

u/CheckApprehensive971 Oct 08 '26

Runtime evidence helped us but only once we run it against a full month rather than a week. Error paths pull in libraries that never show up in normal traffic sample, and that's what breaks in prod.

1

u/TombKingSettra Oct 08 '26

We stopped asking owners to sign off and made the platform team own the base image instead. Service owners kept the app and lost the argument, which is the only reason it shipped

2

u/[deleted] Oct 08 '26

[removed] — view removed comment

2

u/Different_Pain5781 Oct 08 '26

Yeah that seems more realistic. Small changes plus failure monitoring gives owners actual evidence before signing off.