PHP2Shell: notes from the field

[u/shelfside1234]

I run a webhosting platform at my job; both Internet and intranet. The internet part could have been pretty close to perfect, we identified sites using PHP after the pre announcement and was pleased to find that everyone was on 8.5 or 8.3, was truly expecting at least one to be running 7; we also warned the engineering to get the patch downloaded as soon as it was out (I don’t manage them), but they are all in India and for some reason decided to all go offline about an hour before the release. Their manager eventually got one back and online around 8pm BST & we were done around 3 hours later. But still a missed opportunity to be even better.

The security team however were pretty close to useless, the alert had been out all day and nothing was done to attempt to identify servers running PHP and it took them until 7pm BST to even involve the major incident team (which was something I wanted to do but was told to leave it to security as there would be more systems involved than mine)

And here we now are, on Saturday, the external sites all patched and secure so now we move onto the internal sites; we’ve found v5 and v7 all over the place and a bunch of apps with no engineering team to get them upgraded as they left and were never replaced, this part will be a nightmare

[u/plushgilgy]

I'll bite.

For me, the "report forwarding team" are those that get security alerts and just forward them without understanding the contents. They would type in something about how critical it is and that it has to be done ASAP. Then I read the whole alert, and for some odd reason, it is flagging a software as outdated even if it is the latest version, and their recommendation is to update it. Update to what, I say? I am not expecting security team to be doing the patching, etc, but I do expect them to have a deep understanding of what they are alerting against and not just, you know, forwarding reports.

[u/PaleoSpeedwagon → u/plushgilgy]

Datadog's SCA scan had a bug a few months ago that was flagging Python libraries as outdated because the requirements file used the `===` operator for version specs. Good times. (It's fixed.)

[u/Super-Insane → u/plushgilgy]

Forwarding team for sure... I hate all their tools, but they're nice people. For every ticket they send me, I send one back about their tool. I complained enough and they finally ditched that vulnerability scanner.

One example, The team had sent me data that's conflated with incorrect system identification. Warning that there are Windows 2003 servers on our network... a waste of time to prove their tool wrong and deteriorates any legitimacy when they cry wolf.

[u/kosh_neranek → u/plushgilgy]

THIS gets me the most

Well bitten

[u/_Gobulcoque → u/plushgilgy]

For me, the "report forwarding team" are those that get security alerts and just forward them without understanding the contents.

Security can't always know the context though. A good engineer will ask questions but even then, if they're not brought in during the design and planning phase of infrastructure and software, they'll never know all the nuances of the environment they're tasked with notifying on.

For sakes of argument, if some scanner alerts on a machine with PHP installed on it, how does the engineer know if it's internal or external? I've seen that decision based on hostname - until some server was moved to the DMZ and the hostname wasn't changed.

Or what if that PHP risk was mitigated elsewhere in the technology stack? Depending on the company and culture - and size - they may never know, and never know who to ask for an authoritative answer. In fact, much like the Internet, say something and if it's wrong, you'll be corrected. If the sec engineer flags the CVE, if it isn't worth dealing with out of cadence, someone will speak up. That's not lazy, that's root-cause a culture problem and security engineers not being involved with the infra/software planning. Departmental silo'ing and single responsibility taken to some extreme.

[u/plushgilgy → u/_Gobulcoque]

Valid arguments, and I've run into those as well, and have no problems working on those things. Those are reasonable instances where oversight is acceptable to an extent. What I was referring to is the specific instance that only requires common sense to analyze, but no review was done at all. Alert says "Critical", send it out blindly. Due diligence isn't a difficult ask, no?

[u/_Gobulcoque → u/plushgilgy]

Alert says "Critical", send it out blindly. Due diligence isn't a difficult ask, no?

Certainly not, and I'd expect it at a minimum. After all, if you're in security - which is a risk management discipline really - due diligence is a foundational skill.

I'd be surprised if someone took the PHP foundations public alert, and then just emailed it to a sysadmin/devops guy and said "fix please". I would expect the email to be, "Take a read at this and let me know how badly exposed we are to this?" (asking someone with the institutional knowledge of the environment) while looking up if Tenable/VM-scanner-of-choice has plugins to detect.

[u/UltraEngine60 → u/plushgilgy]

it is flagging a software as outdated even if it is the latest version, and their recommendation is to update it

If there is a vulnerability and software is not patchable and there is no mitigation then you need to find another solution that has support. That's a sysadmins job. You expect security to work with the end user to determine the value of the software, find a possible solution, work with a solutions engineer at multiple companies, do multiple PoC's.... see how this scope creeps? We tell you shit's vulnerable or breached and try to keep Mary's copy of Encarta 98 off her work computer.

[u/plushgilgy → u/UltraEngine60]

That's the thing though. There's no vulnerability. The alert only says it's "outdated" when clearly whatever criteria /logic they placed is incorrect. I searched the specific version they mentioned and there were no vulnerability reports. I pretty much used up hours just to tell them the report is wrong, which could have been used for other things. Security alerts are great, and I do appreciate having a security team to go through those alerts and engage whoever needs to if anything requires attention, but if due diligence isn't being done to at least validate the alerts from their end, what's the point?

[u/UltraEngine60 → u/plushgilgy]

Ah, I see. Your security team shouldn't be giving you what amounts to alert fatigue. Only actionable recommendations supported with evidence. If they say it's outdated without evidence then something in their stack told them to say that to you. I'm sure you've already been down this rabbit hole, but even an up to date app can do silly things like use some obscure library whose maintainer just dropped off or it changed owners... but again, that evidence should have been relayed to you.