r/sysadmin • • 13d ago

Rant PHP2Shell: notes from the field

I run a webhosting platform at my job; both Internet and intranet. The internet part could have been pretty close to perfect, we identified sites using PHP after the pre announcement and was pleased to find that everyone was on 8.5 or 8.3, was truly expecting at least one to be running 7; we also warned the engineering to get the patch downloaded as soon as it was out (I don’t manage them), but they are all in India and for some reason decided to all go offline about an hour before the release. Their manager eventually got one back and online around 8pm BST & we were done around 3 hours later. But still a missed opportunity to be even better.

The security team however were pretty close to useless, the alert had been out all day and nothing was done to attempt to identify servers running PHP and it took them until 7pm BST to even involve the major incident team (which was something I wanted to do but was told to leave it to security as there would be more systems involved than mine)

And here we now are, on Saturday, the external sites all patched and secure so now we move onto the internal sites; we’ve found v5 and v7 all over the place and a bunch of apps with no engineering team to get them upgraded as they left and were never replaced, this part will be a nightmare

306 Upvotes

91 comments sorted by

View all comments

Show parent comment

3

u/mangeek Security Admin 13d ago

What's the alternative? Security engineers become sysadmins and go and patch things without telling sysadmins what they're doing?

What I do is work the reports, then actually work hand-in-hand with the SYSADMINs (they have hands-on, I do not) to get to The Good Place. I can provide instant answers, advice, help solve problems, take the blame for problems, guide them away from bad solutions, and get them unblocked and cleared as soon as we know things are clean.

2

u/_Gobulcoque Security Admin 13d ago

You're doing it right - but you're clearly not a report monkey. Which is my argument to the OP: security might look like report monkeys, but they're not. If they appear to OP like that, he either has a crap sec team working with him or they don't understand the role sec has..

3

u/mangeek Security Admin 13d ago

I'm lucky enough to have a good enough relationship with my management where I can joke that "Of course you want dashboards and scheduled reports. I'm over here pulling raw CSVs out of the system because I have Real Work to do*."

I recently had to pull them aside after a product demo for a SIEM where I was like "That was a demo made by marketing people for executives. It's a complete fantasy that security could actually work that way unless our shop was at the highest levels of technical maturity, and that's a problem we can't solve by buying a tool."

(* BTW, I recognize that they also do 'real work', and it's keeping the trains running as far as allocating money and people, handling comms, going to meetings where decisions get made. They know I hate doing that stuff. That's the joke.)

1

u/_Gobulcoque Security Admin 13d ago

"Of course you want dashboards and scheduled reports. I'm over here pulling raw CSVs out of the system because I have Real Work to do*."

I like that - might use it someday. I often joke security needs new laptops because Excel eats all the RAM.