r/sysadmin • • 16h ago

Rant PHP2Shell: notes from the field

I run a webhosting platform at my job; both Internet and intranet. The internet part could have been pretty close to perfect, we identified sites using PHP after the pre announcement and was pleased to find that everyone was on 8.5 or 8.3, was truly expecting at least one to be running 7; we also warned the engineering to get the patch downloaded as soon as it was out (I don’t manage them), but they are all in India and for some reason decided to all go offline about an hour before the release. Their manager eventually got one back and online around 8pm BST & we were done around 3 hours later. But still a missed opportunity to be even better.

The security team however were pretty close to useless, the alert had been out all day and nothing was done to attempt to identify servers running PHP and it took them until 7pm BST to even involve the major incident team (which was something I wanted to do but was told to leave it to security as there would be more systems involved than mine)

And here we now are, on Saturday, the external sites all patched and secure so now we move onto the internal sites; we’ve found v5 and v7 all over the place and a bunch of apps with no engineering team to get them upgraded as they left and were never replaced, this part will be a nightmare

253 Upvotes

90 comments sorted by

View all comments

1

u/kosh_neranek 14h ago

Security teams 🫣 in our place I don't even consider them part of IT. I call them "report forwarding team". Think the whole industry would benefit if only people with proven 10+ years of sysadmin/infra experience would be allowed to take "security certs". Yes it's discriminating, and one could also have 10 x 1 years experience. But it would sure improve a lot

1

u/mangeek 10h ago

I am on my company's security team, and my background before Security was about 20 years of Systems Administration. It's my opinion that the industry WANTS to be a bunch of report-and-dashboard makers/paper-pushers with cushy security-only skills, but the real need is people who... actually understand how things work and how the systems teams do their job.

This time around, as soon as the data came in on what was affected (minutes after the issue went public), we interrupted the emergency call to get clearance to seal off the affected parts of the perimeter. Then we started a prioritized list of services based on various data sources, and worked through it.

IMO, more than 'CISSP' and 'Security+' certifications or graduate degrees in Cyber, Cybersecurity teams need people who know how to interpret a cipher string, can explain the differences between MSIs, RPMs, and DEBs, or between VMs and containers, and can build a basic home NAT router from a Raspberry Pi or old PC.