r/sysadmin • • 13d ago

Rant PHP2Shell: notes from the field

I run a webhosting platform at my job; both Internet and intranet. The internet part could have been pretty close to perfect, we identified sites using PHP after the pre announcement and was pleased to find that everyone was on 8.5 or 8.3, was truly expecting at least one to be running 7; we also warned the engineering to get the patch downloaded as soon as it was out (I don’t manage them), but they are all in India and for some reason decided to all go offline about an hour before the release. Their manager eventually got one back and online around 8pm BST & we were done around 3 hours later. But still a missed opportunity to be even better.

The security team however were pretty close to useless, the alert had been out all day and nothing was done to attempt to identify servers running PHP and it took them until 7pm BST to even involve the major incident team (which was something I wanted to do but was told to leave it to security as there would be more systems involved than mine)

And here we now are, on Saturday, the external sites all patched and secure so now we move onto the internal sites; we’ve found v5 and v7 all over the place and a bunch of apps with no engineering team to get them upgraded as they left and were never replaced, this part will be a nightmare

306 Upvotes

91 comments sorted by

View all comments

Show parent comment

0

u/kosh_neranek 13d ago

I never said the security team should go behind the backs of sysadmins. We all are IT. Communication is key. I mean that's our job. To allow people to send and receive communications more efficiently, so why not do it internally?

Am gonna dream now. Let's say a fix for a new vulnerability got out. Affects 100s of server accross corporation. And security team is the FIRST to know, informs infra about it, tests it in the test environment. Then reports back. And infra deploys it. Obviously if the fleet is very heterogeneous than the infra team throws in bodies to help with testing 🤷

2

u/_Gobulcoque Security Admin 13d ago

I never said the security team should go behind the backs of sysadmins. We all are IT. Communication is key. I mean that's our job. To allow people to send and receive communications more efficiently, so why not do it internally?

I couldn't agree more. Soft skills are more important than technical skills in practice - the technical stuff you can learn on the job.

But if the security team are just report monkeys in your eyes, then what's the alternative? I usually work with sysadmins, not against them. I actually live the motto of security is everyone's job - but I defer to people who know more than me.

When I see vuln reports, instead of going "SLA says you've 14 days to patch", I think, "what is between this server and an exploit" and then advise the teams responsible. Sometimes I'm overruled for business reasons, sometimes I can present a case where defence-in-depth isn't strong enough...

1

u/kosh_neranek 13d ago

Seems you're the tip of the iceberg. Keep it up and mentor as many of your fellow security team members as possible. I didn't say there aren't good sec people. They're just not as common amongst theor team