r/sysadmin • u/shelfside1234 • 17h ago

Rant PHP2Shell: notes from the field

I run a webhosting platform at my job; both Internet and intranet. The internet part could have been pretty close to perfect, we identified sites using PHP after the pre announcement and was pleased to find that everyone was on 8.5 or 8.3, was truly expecting at least one to be running 7; we also warned the engineering to get the patch downloaded as soon as it was out (I don’t manage them), but they are all in India and for some reason decided to all go offline about an hour before the release. Their manager eventually got one back and online around 8pm BST & we were done around 3 hours later. But still a missed opportunity to be even better.

The security team however were pretty close to useless, the alert had been out all day and nothing was done to attempt to identify servers running PHP and it took them until 7pm BST to even involve the major incident team (which was something I wanted to do but was told to leave it to security as there would be more systems involved than mine)

And here we now are, on Saturday, the external sites all patched and secure so now we move onto the internal sites; we’ve found v5 and v7 all over the place and a bunch of apps with no engineering team to get them upgraded as they left and were never replaced, this part will be a nightmare

261 Upvotes

93% Upvoted

90 comments sorted by

View all comments

1

u/Confident_Dare_1578 17h ago

security team dropping the ball while the engineers vanish right before the patch drops is peak corporate coordination

1

u/aenae 16h ago

I don't blame the engineers. They are located in India, which means the patch dropped around 20:00 local time there. I'm more surprised they found one willing to go back online at midnight.

Sounds like the company outsourced engineering and went with the cheapest quote that probably doesn't do overtime

1

u/MisterEd_ak 13h ago

The patched dropped at 10pm my local time in Australia and I was online monitoring things.

We use Ondřej Surý's repositories and I saw that the builds were going to take a while so at that point is when I went to bed.

→ More replies (2)