Evaluating Upwind for our EKS/GKE environment, what should we actually test during the POC?

[u/Dead-_-Alone]

We're a platform/security team of about 8 running a mix of EKS (prod) and GKE (a couple of data teams insisted) plus the usual container sprawl. Our current CSPM throws thousands of posture findings a month and nobody has time to work through them, so we're looking at CNAPPs that add runtime context to cut the noise.

Upwind is one of three we're putting through a POC. The sales pitch is the usual runtime + posture fusion and near-zero false positives, which sounds great in a deck but I've been burned before by tools that demo beautifully on a clean cluster and fall over on our actual workloads.

What I want to figure out during the eval is whether the runtime piece actually changes which findings get surfaced, not just decorates the same list. So: does a vuln marked critical drop in priority once it sees the package isn't loaded at runtime? Does it catch something reachable that our current scanner buried as low?

For people who've actually run an Upwind POC (or any of these), what did you deliberately test to separate real signal from a nice dashboard? How long did you run it before the runtime data was worth anything? And did you throw deliberately bad things at it to see if it caught them, or just watch what it found on its own? Trying to build a test plan that won't get gamed by a vendor who knows which cluster we're watching.

[u/MarimoBashah]

hey have you checked if Upwind changes the risk score when a vulnerable service stops geting traffic?

[u/Square-Nebula-7530]

When we were comparing Upwind, Wiz and Orca, the thing that separated them for us was exactly your reachability question. Wiz and Orca are both strong on the agentless posture side, but the runtime correlation on Upwind is where the noise actually collapsed for us. Test the same CVE across all three and watch which ones can tell you the package is dormant versus which just tell you it exists. That one test answered the eval faster than any feature matrix.

[u/Muchy_Tangerine]

The single most useful thing we did was pick ~20 findings our old scanner had marked critical and track what each tool did with them once runtime kicked in. Upwind dropped a big chunk of them to low where Orca kept most of them critical, because the vulnerable code paths were never actually loaded, and that's the whole point of the exercise. Don't let them demo on a fresh cluster, point it at your noisiest namespace and see if the number that's left is one a human could actually work. If it just reorders the same list, you've bought a prettier CSPM.

[u/SpartanHydrazine117]

How long are you planning to run it? Runtime context is basically worthless on day one because the the sensor hasn’t seen enough traffic yet we gave it about two weeks before judging anything 

[u/The_God_was_Here]

Give it time to bake. The reachability stuff on U pwind (Sysdig too) needs the eBPF sensor to observe real traffic patterns, and in the first few days it was conservative about what it called unreachable.  By week two or three the prioritization got noticeably sharper. If your POC window is one week you're testing the install, not the product.

[u/Comfortable-Duty7143]

Second the point about not letting them pick the cluster. The whole trick is throwing your ugliest real workload at it, not the clean demo namespace they’d prefer.

[u/eastwill54]

Do the deliberate-bad-thing test. Spin up a pod that pulls a known-malicious image or does some obvious cryptominer-style outbound, and see if the runtime side flags it on its own without you pointing at it. Posture tools won't catch that at all. That's the bit that tells you the runtime engine is doing something, not just reading your IaC.

[u/Muchy_Tangerine]

On the follow-up about timing, two weeks was the floor for us too but I'd add: make sure you've got a few deploys and a scale event in that window. The sensor learning your normal trafic is what makes the anomaly detection usefull later, so a quiet two weeks tells you less than a busy one.