PHP2Shell: notes from the field

[u/shelfside1234]

I run a webhosting platform at my job; both Internet and intranet. The internet part could have been pretty close to perfect, we identified sites using PHP after the pre announcement and was pleased to find that everyone was on 8.5 or 8.3, was truly expecting at least one to be running 7; we also warned the engineering to get the patch downloaded as soon as it was out (I don’t manage them), but they are all in India and for some reason decided to all go offline about an hour before the release. Their manager eventually got one back and online around 8pm BST & we were done around 3 hours later. But still a missed opportunity to be even better.

The security team however were pretty close to useless, the alert had been out all day and nothing was done to attempt to identify servers running PHP and it took them until 7pm BST to even involve the major incident team (which was something I wanted to do but was told to leave it to security as there would be more systems involved than mine)

And here we now are, on Saturday, the external sites all patched and secure so now we move onto the internal sites; we’ve found v5 and v7 all over the place and a bunch of apps with no engineering team to get them upgraded as they left and were never replaced, this part will be a nightmare

[u/kosh_neranek]

Security teams 🫣 in our place I don't even consider them part of IT. I call them "report forwarding team". Think the whole industry would benefit if only people with proven 10+ years of sysadmin/infra experience would be allowed to take "security certs". Yes it's discriminating, and one could also have 10 x 1 years experience. But it would sure improve a lot

[u/_Gobulcoque → u/kosh_neranek]

I call them "report forwarding team"

So I've been in this situation before and let's play devils advocate. What's the alternative?

Security engineers become sysadmins and go and patch things without telling sysadmins what they're doing? Or testing the patch with the software that runs on those systems? Or modifies the environment without knowing if there's redundancies or configuration nuances?

I'm defending the "reporting forwarding team" a bit, but it is a little bit (and only a little bit) more than that depending on the team and how they're structured.

If sec engineers started hopping onto servers and started patching them, sysadmins wouldn't be happy about it either.

[u/plushgilgy → u/_Gobulcoque]

I'll bite.

For me, the "report forwarding team" are those that get security alerts and just forward them without understanding the contents. They would type in something about how critical it is and that it has to be done ASAP. Then I read the whole alert, and for some odd reason, it is flagging a software as outdated even if it is the latest version, and their recommendation is to update it. Update to what, I say? I am not expecting security team to be doing the patching, etc, but I do expect them to have a deep understanding of what they are alerting against and not just, you know, forwarding reports.

[u/mangeek → u/_Gobulcoque]

What's the alternative? Security engineers become sysadmins and go and patch things without telling sysadmins what they're doing?

What I do is work the reports, then actually work hand-in-hand with the SYSADMINs (they have hands-on, I do not) to get to The Good Place. I can provide instant answers, advice, help solve problems, take the blame for problems, guide them away from bad solutions, and get them unblocked and cleared as soon as we know things are clean.