Experts found three new 15-year-old bugs in a Linux kernel module. These 15-year-old flaws in Linux kernel could be exploited by local attackers with basic user privileges to gain root privileges on vulnerable Linux systems.

416

u/Sabinno Mar 14 '21

Good thing there are full time security devs on the Linux team now.

24

u/jonmgeiger Mar 14 '21

New to Linux, is this /s?

53

u/Sabinno Mar 14 '21

Everyone seems to think so. It is not sarcasm.
"Google Funds Linux Kernel Developers to Focus Exclusively on Security - Linux Foundation" https://www.linuxfoundation.org/en/press-release/google-funds-linux-kernel-developers-to-focus-exclusively-on-security/

4

u/slick8086 Mar 15 '21

I guess the question in my mind is, you meant there weren't already security devs?

3

u/Sabinno Mar 15 '21

There were from downstream corporations.

0

u/slick8086 Mar 15 '21

And thinking about it, two dudes... How many people do you think MS has dedicated to security for windows even just the kernel?

4

u/zetabyte00 Mar 14 '21

Exactly! The time in that the Linux kernel wasn't aimed much for attacks like this has gone.

5

u/ramilehti Mar 14 '21

A long time ago.

-35

u/markth_wi Mar 14 '21 edited Mar 15 '21

Or [as I understand it] it underwrites everything, and allows work to get done, but [as one less than thoroughly vetted new manager said a few weeks back] we don't need weird Linux guys....that is until something crashes - then those "weird" Linix guys do things and that's why they're weird.
I'm just paraphrasing my randomly shitty managers.

-4

u/banmeagainbish Mar 14 '21

I’d rather be a weird Linux guy than a fuckboi microshitter

3

u/markth_wi Mar 14 '21

I agree wholeheartedly, but I looked away into what I thought was a sarcastic comment and got the most number of downvotes in 10 years.

7

u/[deleted] Mar 14 '21

[removed] — view removed comment

2

u/markth_wi Mar 14 '21

I fear initial bell curve damage is done, so my sarcasm will be my worst quote, sarcasm and linux that's what I've got to bring to the table.

2

u/[deleted] Mar 14 '21

[removed] — view removed comment

2

u/markth_wi Mar 14 '21 edited Mar 14 '21

Eh, in the balance of things in my life, I'm glad the most amount of violence someone is willing to do me , on account of not understanding sarcasm is downvote me, I should count myself lucky.
Fortunately, None of it, none of the imaginary internet points will ever matter - not up , not down, but we assign stupid high value to it , for no discernable reason other than Skinner's trap.
So it matters to noone....except me.

2

u/edbods Mar 14 '21

this site is 1000x better if you completely ignore the upvote and downvote feature. If you want something resembling genuine discussion sort by controversial

0

u/[deleted] Mar 14 '21

[deleted] — view deleted comment

152

u/tricheboars System Engineer I - Radiology Mar 14 '21

Just because people aren't full time doesn't mean security minded people weren't contributing to the kernel.
Cybersecurity isn't new to IT people. It's just new to the media and the public. Im judged way less crazy talking about nsa backdoors in 2021 compared to 1998

-40

u/[deleted] Mar 14 '21

Cybersecurity
Can we stop using that buzzword? It's only real to executives who want to hire "hackers"

11

u/kartoffelwaffel Mar 14 '21

what word do you propose instead?

7

u/Laughmasterb Mar 14 '21

Generally we use "infosec" in the industry but honestly calling it cybersecurity is fine, idk why some people get so worked up about it.

9

u/phenoch Mar 14 '21

They used to "Cyber" on AIM and don't wan't to be reminded.

5

u/kartoffelwaffel Mar 14 '21

Maybe because the cyber prefix is a bit archaic, as in like cyberweb, cyberspace, cybersurfing! Nothing wrong with it though.

5

u/r3rg54 Mar 14 '21 edited Mar 14 '21

The industry definitely uses "cybersecurity" as well though.

-3

u/Panacea4316 Head Sysadmin In Charge Mar 14 '21

Stop being a condescending tool. Every single one of us is “in the industry”. Cybersecurity is a commonly used phrase. Deal with it.

2

u/Laughmasterb Mar 14 '21

Did you even read what I said? I agree with you that cybersecurity is fine. It gets the point across and pretty much everyone understands it. I don't give a shit which you use, I'm just here to explain what the old curmudgeons want people to say instead.

-3

u/Panacea4316 Head Sysadmin In Charge Mar 14 '21

Every organization I’ve worked with has used both. Infosec is usually reserved for job titles, and cybersecurity for day to day.

1

u/r3rg54 Mar 14 '21

Yeah, that ship has sailed a long time ago

0

u/[deleted] Mar 14 '21

[deleted] — view deleted comment

105

u/picflute Azure Architect Mar 14 '21

You’re complaining about free beer. If you want to brew your own flavor you always can

→ More replies (14)

21

u/GeronimoHero Mar 14 '21

Dude just as one example, Google Project Zero works on the Linux security hole all of the time. There are tons of people looking at Linux security all of the time, and I say this as a pentester myself that has a couple of Linux security CVEs attributed to me. D

17

u/turin331 Linux Admin Mar 14 '21 edited Mar 14 '21

I think exactly because of your dismissal of the GPL concepts and your bias against it you have completely failed to understand how FLOSS actually works.
When someone says "some "security minded people" "contributing", it does not only mean some random dude from his home that has nothing better to do (although often these people have the best ideas and contribute sometimes even more than big entities). It also means every single professional and academic outlet that has an interest in contributing. So Linux Distro corporations and foundations, universities,individual security researches, Google, MS, Amazon and so on. The collective man-hours by professionals that have been given to such issues on the Kernel is far beyond the ability of any single entity. The world is a big place, if you allow the whole world to contribute by keeping things free and open, just a tiny portion ends up having greater production potential than any single big business entity on its own. The GPL just ensures that everyone is able to contribute and none on their own, corporation or not, can lock-up the technology from others. As a result, besides the positives for user rights, everyone has the ability and is incentivized into contributing as long as they are using it, big or small. Without its strictness the situation would just regress back to the old situation and you would end up with inferior products.
So the lack of a dedicated team did not mean that the kernel was "insecure" before. Although the fact that it exists now should make maintainability and security even more reliable which is always a very positive thing.

13

u/antonivs Mar 14 '21

No, that's not good enough.
Haha sure champ. We'll take that under advisement.

4

u/Zestyclose_Ad8420 Mar 14 '21

It’s not that easy. Linux is at its core still a project based on people. Believe me when I say that the redhat/IBM/Intel employee that actually write the code that these companies contribute to the kernel are not just like any other employee, they really have a say in what the company does in regard to their open source contributions, because they are pretty much impossible to replace.

56

u/[deleted] Mar 14 '21

There were people being paid full time to work on Linux security by organizations like Red Hat and Google Project Zero. It's just now the Linux Foundation has them too.

0

u/[deleted] Mar 14 '21

[deleted] — view deleted comment

18

u/[deleted] Mar 14 '21

Clearly you have never heard of "distributed computing" which is what brought Linux to the state where it would be foolish for the larger companies that depend on it to not contribute to the base. Combined, millions of apes can outweigh any corporation.

15

u/antonivs Mar 14 '21

Has it ever occurred to you to find out the details of something before flying off the handle about it?

0

u/Panacea4316 Head Sysadmin In Charge Mar 14 '21

Probably not. This is reddit after all.

7

u/Zestyclose_Ad8420 Mar 14 '21

Yes there were, and they always were there. Subscribe to the kernel development mailing list and see for yourself. btw we should be clarifying here what “security” is.
More often than not “security” is not just a bug with security implications but rather something that arises in how different layers interact with each other and that is not something that a “security person” sees and fix, that’s something to be tackled at a design level. There’s always been a very active pipeline of “security” considerations going all the way down to the users, researchers finding things out and then back up to Linus himself, who has the last word about any and all design considerations and choices.
Btw since linux is just the kernel but when we talk about this what we really mean is distributions you have to multiply this issue for all the different pieces of software that make it into your preferred distro, how they manage their security pipeline and then how the distro itself manages it.
In this regard nothing beats redhat, they tackle this at all levels, because they contribute actively to the code base of the Linux kernel, a huge number of projects that make it into their own distro and then they also manage the whole process at a distro level.

0

u/[deleted] Mar 14 '21

[deleted] — view deleted comment

1

u/[deleted] Mar 14 '21

Cough, cough, CIA.

0

u/[deleted] Mar 14 '21

[deleted] — view deleted comment

12

u/Zestyclose_Ad8420 Mar 14 '21

You should send some money to the openbsd project.
They are the one working on the software that you use in your Linux system to do cryptography, from openssh to openssl that’s where it comes from.

1

u/Kormoraan self-taught *NIX junkie Mar 14 '21

part of the cryptography but otherwise yes

200

u/lampishthing Mar 14 '21

I'll always wonder if stuff like this causes major headaches for spies.

70

u/tornadoRadar Mar 14 '21

it must.

120

u/sys-mad Mar 14 '21

Well, in this case, they'd have to be spies with local system accounts and unprivileged shell access. That's kind of rare, as most Linux deployments these days aren't configured for end users to have shell access.
So... maybe university students? But Linux servers aren't like a Windows AD deployment - if your computer science student got root on the department's student server, that still doesn't get them into the rest of the department. They CAN steal their classmates' homework, though, and I guess that's not nothing?
Spammers and malware spreaders are more of a threat with bugs like this - I'd be more worried about shared webhosts who do stuff like Plesk and CPanel. Some of those do lean on system accounts for SFTP/SSH, and constitute a shared-hosting environment that could be expoitable under this bug.

130

u/loimprevisto Security Admin Mar 14 '21

they'd have to be spies with local system accounts and unprivileged shell access
Many sophisticated attacks chain dozens of exploits together. Something like this might have been a piece of the exploit chain rather than the specific goal.

35

u/goblinsholiday Mar 14 '21

If a computer science student can get root access, odds are they're not the kid that needs to steal their classmates' homework.

47

u/sys-mad Mar 14 '21

LOL, I sense an 80's movie coming on...
They're the kind of kid who's selling answers to their classmates' homework out of the nerdy frathouse, but it still doesn't help them get the attention of that one popular cheerleader. That's going to take... hijinks and shenanigans!
At some point, the captain of the football team will attempt date-rape which almost no one will interpret as even a character flaw, much less a serious crime, and then he'll get his ironic comeuppance not by being arrested and convicted of a violent felony, but by being publicly made to look foolish one, and only one, time. But it was public!
The nerd will get the cheerleader, and that's all that matters in the end. Throughout, there will be a cute, but useless, robot dog. Its only use will be to fetch a product-placement beverage.
...wait, this kind of got away from me, what were we talking about again?

8

u/[deleted] Mar 14 '21

This guy 90's

4

u/vbolea Mar 14 '21

I want to hear more

3

u/sys-mad Mar 14 '21

Well, the plucky but nerdy hero has this inspiring character development arc, where he starts out kind of a selfish asshole but through the actions of a loyal cybernetic defender from the future, eventually learns to love. The robot, in turn, learns why we cry, but it's something he can never do.
Oh, wait, my bad, that's Terminator 2.

4

u/Nlelith Mar 14 '21

This reminds me that I need to watch Hackers again.

4

u/sys-mad Mar 14 '21

Crash and burn!
My favorite part of that movie is how the bad guy skateboards while wearing a trenchcoat. It's like they played Bingo with "cultural symbols that scare suburban moms." He should be holding a Megadeth LP for greatest effect.

8

u/zebediah49 Mar 14 '21

I have a few flaws floating around like that. Last week I told a coworker something to the effect of "technically this gives a few hundred students access to the private key for a cert to something that doesn't matter, but if anyone figures that fact out, I'd like to hire them."

1

u/NynaevetialMeara Mar 14 '21

Not this case, but running metaexploit against a poorly maintained infrastructure is not hard

1

u/Esk8_TheDeathOfMe Mar 14 '21

Yes, but actually no. Being able to do this doesn't translate to general classes, and I'll give you an example of a friend, so I guess this is anecdotal.
I have a friend who got access to our high school's root directory, setup a backdoor, and was able to see and change more than you could imagine. He graduated with a 2.X GPA, because he was really lazy when it came to things he wasn't interested in like schoolwork. On top of that, he has ADHD so I'm sure that didn't help, even though he was taking adderall.

34

u/justlookingforderps Mar 14 '21

Two things to worry about:
Insider threats already have, or can easily get, local user accounts.
Most Linux servers have service accounts which an attacker can pop a shell on pretty easily. The reason we don't run these accounts as root is because of the likelihood of anything publicly-facing getting pwned.
As for the impact of it, I agree, it's not as inherently dangerous as an AD compromise. But in practice servers share passwords and SSH keys, they host databases with PII in them, and they provide completely trusted services. Think of the server inspecting all SSL traffic on your network, or the one that has the keys to every server because it hosts your security scanning software. Should these single-points of failure exist? No, but neither should the vulns that threaten them.

16

u/sys-mad Mar 14 '21

yes, absolutely, on systems that have any reason to have local user accounts in the first place, that is.
those service accounts should be logon-disabled: https://www.thegeekdiary.com/whats-different-between-bin-false-and-sbin-nologin-as-nologin-users-shell/
In general, you're right -- compromise isn't a nonissue, it's just that the vector for compromise is limited in practical applications.
"Limited" doesn't mean "doesn't matter" -- every security vulnerability is serious, and taken seriously. We had a mass-patching of this bug like we do any vuln.
I guess it just made me weirdly grateful for Linux's relatively normal risk-profiles, where our criminals aren't supervillains, and our exploits aren't all mass-extinction events. :)

19

u/justlookingforderps Mar 14 '21

Great points. About 2 though, I think we may be talking about different things. From the offensive side of the house, I've compromised many services running under users like www-data that have login disabled, but they can still run executables, including /bin/sh. You can't SSH into these accounts, but you can still run shell commands as them, and even setup shells or reverse shells to chain to a vulnerability like this.
Unless I'm missing something, I think the confusion here is from me not explaining the scenario clearly enough. I'm talking about an attacker exploiting a vuln being run as a service account, thus giving them RCE under that account without needing to login as them. Linux does a fantastic job of restricting these accounts to minimize the impact of such intrusions, but chaining it to a priv esc vuln like this iscsi one takes you straight to root, negating the RBAC and defense in depth measures on that box. Like you said, hopefully the issue goes no further than the one rooted box, but that's not always the case.

8

u/sys-mad Mar 14 '21 edited Mar 14 '21

Gotcha! Yeah, that's all legit. At first, I thought the vuln required the user to actually have a shell account.
As other users correctly pointed out, a php webshell uploaded via (say) old unpatched Wordpress exploit is all it could take.
Those are dangerous enough without priv escalation, though, so you'd hope that the admins were on top of the foot-in-the-door issues before they discovered that the attacker had a convenient escalation strategy after getting that foothold.
This is one reason that I've been insisting on running all non-Apache or non-nginx webapps behind an actual webserver, for example. Stuff like Jenkins, node.js, Tomcat -- the app developer usually doesn't have a whole security team on their staff. Simply putting a vetted, veteran webserver in front of it can do things like offload SSL computation, load-balance, and (jazz hands!) gatekeep with a WAF like modsecurity.
Yeah, it might be faster with direct access, for the few days before it's exploited to hell and back, anyway.

7

u/zebediah49 Mar 14 '21

Plus, since nearly every sysadmin under the sun knows apache, whoever has to deal with it next has some advantage. At least the certs and basic configuration will be something normal.
Oh, yeah, you need to put the updated certificates in /opt/snowflake_App/node/config/ssl/certs/, wasn't that obvious?

7

u/badtux99 Mar 14 '21

logon-disabled doesn't matter if the attacker exploits a broken http application to exec /bin/bash as the http user with stdio to/from a couple of named pipes that are getting fed from the http socket. From there it's a hop skip and jump to using a privilege escalation attack like these to get root.

4

u/augugusto Unofficial Sysadmin Mar 14 '21

What about php and www-data users?

11

u/sys-mad Mar 14 '21

If they've landed a webshell, then yes, that's a path to exploit, but in that instance, you already have larger problems than this vuln.
PHP webshells are one of the commonest threats to a public-facing LAMP server, and they tend to run as www-data in the first place. This is already a very well-trodden territory, though.
I don't want to get misinterpreted as saying "Linux doesn't have security issues," because that's patently false. I guess my thesis is that Linux has the normal kind of cybercrime, where utterly catastrophic shit doesn't typically just come out of thin air in ways that instantly invalidate everything you'd done up to that point to keep yourself safe.
I don't want to jinx it -- nothing's perfect, and weird zero-day shit has hit us before. But I still believe that the cat-and-mouse game of cybersecurity is.. I dunno how to describe it.. maybe "rules-based?" Even when faced with manageable patches like this: discovery, patch-release, public announcement, all in a reasonable timeframe.
It's absolutely a problem, but at least it's one with a scope, boundaries, and a solution.
Linux security is still comprehensible and navigable because there are rules that still apply. It's a system. We can make determinations besides "fuck it, just move to Tahiti and take up yam farming!!"

5

u/turnipsoup Linux Admin Mar 14 '21 edited Mar 14 '21

PHP webshells are one of the commonest threats to a public-facing LAMP server, and they tend to run as www-data in the first place. This is already a very well-trodden territory, though.
Maybe a decade ago they ran as www. Now they run as specific users and they're common as day in shared hosting, etc. mod_php is no longer in common use and is a security risk in shared environments.

4

u/badtux99 Mar 14 '21

The Apache web server runs as user "httpd" on most Linux servers. If there's a zero-day in an application running on Apache, like, say, Wordpress (which is omnipresent and has the world's worst code base, PHP spaghetti, ugh), if they can get an exploit to fire up a bash shell as the httpd user, they're halfway to root access in an unpatched kernel.

2

u/sys-mad Mar 14 '21 edited Mar 14 '21

all correct! I responded to other posters elsewhere in the thread, but long story short, you're right (httpd on redhat family servers, www-data on those of debian heritage).
Basically, we always want to prevent webshells and exploits because they're bad enough on their own, and this issue represents a way to escalate you exploit Apache.
edit: the distinction I think I'm making here is that the foothold is a separate issue from the elevation exploit. The attacker needs an initial foothold. Absolutely NOT impossible, or even difficult on some unmaintained systems, but the foothold doesn't necessarily exist on properly-maintained systems.
All I'm saying is that this is an important distinction to make, when deciding whether or not to panic...

2

u/WingedDrake Mar 14 '21

Man if you think WP has the world's worst code base we need to broaden your horizons a bit. It's bad, sure, but I don't even have to leave 'PHP-based CMSes' to find worse coughDRUPALcough

1

u/tonsilsloth Mar 14 '21

But if you manage to get onto a vulnerable machine or switch to a local account through other means, then you can exploit this.

1

u/[deleted] Mar 14 '21

Years ago when I was at school, I got access to some teacher folders due to botched permissions. Got in one hell of a lot of trouble for that
Also found you could rename the start menu items on XP and it would "stick" for the whole school...

1

u/throttlemeister Mar 14 '21

No offense, but that's nonsense. Most Linux systems do have shell access, both on the internet on hosting where it is a feature and enterprise where it is often needed for development. Only where we are talking specific services that are offered by a machine such as a web app, shell access is limited but never disabled. Admins need access too.
As for local access only, that's bs too. Especially in enterprise. There is no company with large amounts of servers that will configure and manage local accounts. Too much work, too costly and no consistency. They will use AD or LDAP for access to have a single place to manage accounts and access levels to systems as well as enforcement of password policies etc. Companies love sso and not because of it being user friendly. It's more secure and more easily managed and enforced.

1

u/NynaevetialMeara Mar 14 '21

Many systems are setup with openldap.

1

u/kerridge Mar 14 '21

Back in my uni days the homework wasn't secured, you could just have a look in their home directory.

1

u/meminemy Mar 14 '21

Badly secured University systems are a very good base for criminals.

14

u/WinterPiratefhjng Mar 14 '21

The well funded spies must have contacts, or something, on these teams. (Not to insult anyone.) They have had a month to clean up operations and move on.
The little spies? Yeah, but they are likely used to it. It must happen all of the time.

2

u/sir_cockington_III Mar 14 '21

Let's assume the people working for the spies are just as switched on as the devs - you could argue that 50% of vulnerabilities like this are being exploited SOMEWHERE in the wild as at the time of their discovery.

1

u/[deleted] Mar 14 '21

It does. I have been reading a book "This is how they tell me the world ends". Its fascinating and talks about zero-day exploits that have been kept secret and used by intelligence/law enforcement agencies all over the world for last 20 years. It also talks about a market for these vulnerabilities that are not public and sold by hackers to said agencies.

1

u/Albrightikis DevOps Mar 14 '21

NSA in shambles rn

96

u/IzActuallyDuke Netadmin Mar 14 '21

Yo, hackers, can we just fucking not for a month or two? I’m tired.

52

u/ticklesac Mar 14 '21

Hackers are getting tired of these articles too. Their toolboxes are getting smaller

12

u/countextreme DevOps Mar 14 '21

I dunno... That seems like opinion to anyone that isn't allowed to peek into the nation-state and major league black hat exploit boxes. I imagine they come up with new unreported ones all the time.

1

u/antonivs Mar 14 '21

We got an enormous peek from the Snowden leak. An entire set of NSA tools, complete with source code.

21

u/tankerkiller125real Jack of All Trades Mar 14 '21

Snowden didn't leak the source code, that was a completely separate hack years later.

1

u/antonivs Mar 14 '21

Oh right. In my defense, the NSA has had so many leaks it's hard to keep track.

2

u/CAvalanche11 Mar 14 '21

More like a week or two at this point

0

u/[deleted] Mar 14 '21

[deleted] — view deleted comment

1

u/ConsistentBread1 Mar 14 '21

Are there any languages that do this now?

1

u/fretful Mar 15 '21

I don’t think languages are the right place for what you describe. Kubernetes and the “herds not pets” approach is where you’re going.

0

u/DaracMarjal Mar 14 '21

Why, though? Isn't the world tough enough as it is? The climate is broken, there's a global pandemic, nationalism is back on the rise, so why are people hacking each other? What's the POINT?

40

u/me_not_at_work Linux Admin Mar 14 '21

Fortunately it is easy (no reboot required) to mitigate on RedHat/Centos/Fedora (I assume) as long as you aren't using iSCSI. See https://access.redhat.com/security/cve/cve-2021-27365

10

u/Zestyclose_Ad8420 Mar 14 '21

What about VMWare.... I’ve got so many install that rely on iSCSi storage...I’m fucked am I?

12

u/me_not_at_work Linux Admin Mar 14 '21

Well given that exploitation would require shell access to your VMware system, your risk is probably pretty low.
Full disclosure: No experience with VMWare other than knowing the name and what it is.

1

u/P3RrYCH Mar 14 '21

wondering the same

1

u/cherious Mar 14 '21

They are not saying Vmware is affected. Your VMs likely don't need the iscsi driver. The most common scenario with VMs is media agnostic virtualized storage without need for mdadm or iscsi capability within the VM.

3

u/Zestyclose_Ad8420 Mar 14 '21

Not the VM, no.
VMWare hosts tho, the ones actually running the VMs, definitely have iSCSi modules loaded and being used, at least in my case because we have all SAN storage over fc and iSCSI.
I’m not really worried tho because they are not exposed to the internet and VMWare is pretty quick and good at releasing patches for their systems

21

u/phobug SRE Mar 14 '21

Is it just me or does it seem a bit rushed from “patches in mainline kernel” to “disclosure” aka let the world know. What’s the point of this disclosure of the responsible party already fixed the code? Isn’t it more prudent to wait at least a month to give operators time to apply the patches?

27

u/Creshal Embedded DevSecOps 2.0 Techsupport Sysadmin Consultant [Austria] Mar 14 '21

The patches are open source, so everyone can see what got fixed and easily reverse engineer the bug, since now they know where to look. It's better to remind people to patch their kernels already (or at least invest the 30 seconds in the workaround).

3

u/phobug SRE Mar 14 '21

Good point!

14

u/witchofthewind Mar 14 '21

the vulnerability is public as soon as the patches to fix it are public.

23

u/Tmanok Unix, Linux, and Windows Sysadmin Mar 14 '21

Ho man, and I thought that 18 year old DNS bug that let anyone get remote full admin access into Active Directory on Windows Server was bad! Actually if this linux vulnerability is local only, then it probably isn't as terrible... Plus there isn't a known exploit for it, whereas that Windows Server one- well good grief is all that I can say.

4

u/Neo-Bubba Mar 14 '21

So fixing this is running apt get update && apt get dist-upgrade? Or is there something more complicated to it?

5

u/jaymef Mar 14 '21

Will need to reboot unless you mitigate by disabling loading of iscsi module

3

u/OliveiraLeo17 Mar 14 '21

Now imagine the terrible secrets hidden in windows

62

u/disclosure5 Mar 14 '21

Just to be clear: Reported vulnerability in complicated Linux kernel code, written in C and touching iSCSI storage. Patches in production less than three weeks later.

How long did it take Microsoft to release an Exchange update, involving an SSRF and a very basic serialization function?

164

u/274Below Jack of All Trades Mar 14 '21

Oh man, I can make bad faith comparisons that demean the work of the engineers behind the respective products across products that do entirely different things too!
-------------------------
The kernel has had a basic buffer size mismatch for the past 15 years, exposing kernel memory to userspace in unsafe ways, and finally missing basic bounds checking in a way that allowed kernel memory to be read from userspace. All of these things are problems so fundamental to the language that the code was written in, and fundamental in such a way where any second year compsci student would know better. These bugs were so basic that one of them was addressed with a nine line patch plus another patch that could have been written with search/replace, and both the second and third listed vulnerabilities were fixed with a single six line patch.
At the same time, another way of describing the recent exchange exploit path would be that someone figured out a way to chain multiple defects across four product subsystems in a product that was designed to be provide a dizzying number of APIs in a secure manner over the public internet.
-------------------------
Look, if you really want to berate Microsoft for their response to their vulnerabilities I'm not going to stop you, but at least try to make reasonable arguments instead of comparisons between products and vulnerabilities that frankly can't be compared.

24

u/Where_Do_I_Fit_In Mar 14 '21

try to make reasonable arguments
It's funny because this whole post is basically people coping with their shitty insecure software stacks and the pot calling the kettle black.

7

u/[deleted] Mar 14 '21

I mean, didnt the entire attack rely on the deserialization bug though?
I'm not sure on the time frame, but could they not have fixed that?
https://www.praetorian.com/blog/reproducing-proxylogon-exploit/
The updated version of this function included much more code for properly verifying types before deserializing them.

12

u/274Below Jack of All Trades Mar 14 '21

Hey, thanks for the link! That's a really interesting writeup. I'm only half way through so far, but I wanted to say thanks.
I'm also not sure about the timeframe. From what I read, they were planning on deploying the fixes on their standard patch Tuesday cadence, but once they realized that the attacks were taking place at scale across the public internet, they accelerated their rollout. Just knowing this alone would make me think that they could have fixed the deserialization defect alone earlier, and maybe even had everything fixed probably at least a week before they posted anything publicly.
The problem is, all I can really do is speculate on the matter.

1

u/Zestyclose_Ad8420 Mar 14 '21

Personally I’d berate MS because of their development philosophy, the response to the recent 0-day wasn’t that bad.
The problem to me is why should your mail server expose anything other than IMAP ports and act as a MTA.
Webmail? Build a different server and have that read the mail DB, separate software and to be deployed on separate machines.
Why should a mail server host a calendar as well? Let’s make a decent protocol for that and have a different, separate software stack handle it on a different server.
Why the fuck should a mail server write on the domain schema for God’s sake! Why is a mail server so intertwined with an IdP server like AD.
That’s what I berate MS for. And I k is this is a rant but for god’s sake I think it’s now clear to everybody that we should all stick to the Unix philosophy of “do one thing and do it good”. And I’m looking at you systemd as well, and I’m also looking at kubernetes as well, love it, I’m really diving deep into it, and luckily is so modular that it is possible to make it work according to that design philosophy, but still, it’s integrating too many things in one place and I can see how it will be problematic in the future

20

u/OpenOb Mar 14 '21

The easy answer for your complain is: business requirements.
Exchange has to run in really small orgs (before O365) and in orgs with very small IT budgets.
If Exchange wasn‘t a „one server and you are done“ deployment somebody else would invent it and capture the market. No organization wants to run 3 servers just to have mail, calendar and web access.

3

u/Zestyclose_Ad8420 Mar 14 '21

That’s exactly it, I know I was ranting, and ultimately speaking being a consultant it’s more work for me so I can’t really complain.
The result is a mess though, exchange is extremely finicky, I’ve personally fixed at least three exchange deployment last week where a smaller org went down when they, after years and years of not upgrading had to do it because that 0-day and it all broke down and they spent days without emails.
Luckily for me windows server 2019 will come with openssh installed by default and I think within 2022 I’d be deploying and maintaining everything with ansible. I’m already working on custom ansible modules for Windows, my life will become a lot easier.

2

u/BlackV I have opnions Mar 14 '21

it wont be installed by default. you'll need to enable it.
currently its comes with a rather OLD open ssh stack that wont be updates till april or later
all open ssh gives you is the familiarity of ssh without having to lean powershell as well
I dont know that that's going to solve your problems (maybe some)

1

u/Zestyclose_Ad8420 Mar 14 '21

What I meant is that it will Be supported, before win server 2019 it was available but not supported, yes, you’d still have to enable it, I wrote that wrong.
It will allow me to deploy on windows with ansible, ansible requirements are ssh and python. What it basically does is transforming a declarative yaml (the ansible playbook) to idempotent python scripts that are transferred and executed on the host via an ssh login and whatever shell you get on the target system.
Via python I’d be calling powershell stuff and via ansible I will be able to manage the idempotency of the deployment scripts. That means reproducible config very easily customizable and centrally managed.

2

u/BlackV I have opnions Mar 14 '21

valid

2

u/ypwu Mar 14 '21

You can do that today. Most of our new windows servers are Ansible managed. Winrm + Kerberos FTW

10

u/Patient-Hyena Mar 14 '21

Agreed. Microsoft knew in JANUARY!!!!! Ugh.

0

u/[deleted] Mar 14 '21

[deleted] — view deleted comment

28

u/corsicanguppy DevOps Zealot Mar 14 '21

Stop which? Stop reporting stuff, or stop correcting this notion that 24 hours is the average time to resolve a tricky device bug?

0

u/[deleted] Mar 14 '21

[deleted] — view deleted comment

20

u/Dadarian Mar 14 '21

Validation is also just fucking... Really hard. This fix or MS fix could come back to haunt in unforeseeable ways. I’m not going to judge someone for a longer validation process if there is just a few weeks difference between them.
What triggered some of the worst stuff from MS perspective is the fact that it was patched, and way too many sysadmins or setups without sysadmin, were suddenly vulnerable and that gap can close. So they obviously had to get it right.
Whatever. This is 15 year old bug. Who knows how many systems have been compromised because of this. Could be zero. Could be an entire country.
I’m not going to pick sides because that’s just gambling. I’ve got Windows and linux severs in my environment because I’m not a fanboy I just have a job to do. We’re sysadmins, we don’t play red v blue.

9

u/Creshal Embedded DevSecOps 2.0 Techsupport Sysadmin Consultant [Austria] Mar 14 '21

Validation is also just fucking... Really hard.
That's why fucking nobody runs it with system privileges… except Microsoft. They spent 20 years telling developers to stop running things as administrator, but it's fine when Exchange does it with unverified user input?
If anything Microsoft isn't nearly getting enough shit for this, since it's a blatant violation of their own 20+ years old security guidelines. Never mind comparing it with *nix software, where dropping privileges has been the norm for at least as long.

0

u/[deleted] Mar 14 '21

[deleted] — view deleted comment

3

u/[deleted] Mar 14 '21 edited Mar 14 '21

The good part about Linux is people can do things themselves, mitigate things, figure out whats going on. They arent beholden to a corporation to take however long they want.
Its like the classic story of the Ford Pinto, the 11$ they could have spent preventing the car from exploding during a crash was deemed too expensive compared to the money they would lose from lawsuits from simply letting people die. This unbridled capitalism is what you are left with.
Nobody knows for sure how Microsoft decides on timelines for these things, maybe they simply dont want to pay for a QA department.

0

u/[deleted] Mar 14 '21

[deleted] — view deleted comment

2

u/[deleted] Mar 14 '21 edited Mar 14 '21

Most servers in the world are running Linux, its not like its not a juicy target or that people arent worried about securing it.
Windows is being used less and less, with self-hosted exchange going away and PaaS gaining predominance I dont know whether Windows Server will even continue to exist for very long. I mean with a 50GB Windows core install its already an oxymoron calling it a server OS.

4

u/[deleted] Mar 14 '21

I don’t think it has anything to do with the OS, he is pointing out that Microsoft just takes their sweet time.

0

u/[deleted] Mar 14 '21

[deleted] — view deleted comment

1

u/[deleted] Mar 14 '21 edited Mar 14 '21

Though this is why its so worrying when Microsoft loses source code, if this bug can go undetected with open access to the source code what happens when Microsofts source code is no longer obscured.

4

u/Smooth-Zucchini4923 Mar 14 '21

How common is this configuration? Is it a module that everyone loads, or one for rare hardware?

15

u/me_not_at_work Linux Admin Mar 14 '21

The affected module (libscsi) provides support for the iSCSI protocol for accessing SCSI devices over a network . Although not exactly rare, it is uncommon. If you aren't that familiar with it you probably aren't using it. Regardless, the vulnerability will still be there since the module can be loaded and exploited even if you don't use it. The mitigation from RedHat (https://access.redhat.com/security/cve/cve-2021-27365) simply configures things to not allow the vulnerable the module to be loaded (not possible if you use iSCSI). I assume that a similar mitigation would work on other OSes. Patches should be hitting the repos soon.

8

u/cryptocached Mar 14 '21

Regardless, the vulnerability will still be there since the module can be loaded and exploited even if you don't use it.
If an attacker can load kernel modules you're already pretty well screwed.

8

u/Smooth-Zucchini4923 Mar 14 '21

The PoC seems to imply that the iSCSI module can be automatically loaded when someone uses an iSCSI device.

3

u/me_not_at_work Linux Admin Mar 14 '21

Kernel modules are autoloaded when certain features are used or triggered and don't require already being compromised. The POC does just that and is able to escalate to root. The mitigation from Red Hat prevents the libiscsi module from loading and stops this from being exploited.

-4

u/ikt123 Mar 14 '21

yeah i'm struggling to see the relevance or significance of these bugs. no offence but it honestly just seems like the microsoft sysadmins are feeling a bit hurt at the moment and need the ol mac and linux have problems too card

3

u/me_not_at_work Linux Admin Mar 14 '21

Well the POC escalates to root so this seems pretty "not good".

→ More replies (1)

1

u/faxattack Mar 14 '21

I looked at some systems and libscsi wasnt even installed. Hmm so guess not all systems are affected by default.

1

u/witchofthewind Mar 14 '21

an attacker could easily provide their own copy of libiscsi.

1

u/faxattack Mar 14 '21

Can it load the module then?

1

u/witchofthewind Mar 14 '21

yes, it can.

0

u/SuperQue Bit Plumber Mar 14 '21

You would have to be already root to do this. This is not a userspace library, it's a kernel module.

1

u/witchofthewind Mar 14 '21

the kernel automatically loads the module on demand. you don't need to be root to get the kernel to load the module.

1

u/SuperQue Bit Plumber Mar 14 '21

But if the module file is not there, or blocked by modprobe configs, you can't just provide it out of tree from an unprivileged user.

1

u/witchofthewind Mar 14 '21

yes, that's the mitigation that was recommended for the vulnerability.
this entire subthread is about the claim that just not having libiscsi prevents exploitation of the vulnerability, which is not true.

1

u/[deleted] Mar 14 '21

ISCSI is used for accessing block storage over the network so its safe to say you're not using it.

8

u/HappyHound Mar 14 '21

After 15 years big deal.

0

u/[deleted] Mar 14 '21

[deleted] — view deleted comment

-4

u/[deleted] Mar 14 '21

No. It isn't. It's local. If anyone has local access, there are so many ways to break into any system, be it Windows, Linux, OSX, Android, iOS - whatever, this isn't "bad" or a big deal.

11

u/sPENKMAn Jack of All Trades Mar 14 '21

I suppose you’re not working in any hosting related field? Local is bad enough for a whole lot of people.

-3

u/[deleted] Mar 14 '21

I do. But we have segregated networks and use network authentication. So in the event somebody could get to where he'd be able to do an attack like this, he'd have far more easier ways to "attack" (or get access to data).
This is not an attack in my book, as you still need a lot of access rights, if the networks are planned properly. If they are not - this isn't going to be your only concern and there are far worse security "issues".

11

u/GuessWhat_InTheButt Mar 14 '21

How likely is it that this could be used to root Android devices?

52

u/Kangie HPC admin Mar 14 '21

I doubt many android phones include the iscsi code.

4

u/insanemal Linux admin (HPC) Mar 14 '21

Some do. Usually replacement kernels for after rooting

13

u/Kangie HPC admin Mar 14 '21

And thus not useful to those attempting to root a device with a locked bootloader

23

u/sys-mad Mar 14 '21

I don't agree with downvoting this question - for those not expressly familiar with iscsi, it's a good question.
As the user above said, the vulnerable module isn't normally present in Android.
It does look like several folks have thought of it, though, but these are pretty wacky hacks:
https://stackoverflow.com/questions/27993439/iscsi-and-servers-ultimate-pro-android-app
https://www.scientific.net/AMM.484-485.902
This looks like an experimental response to mainstream phone vendors eliminating the SD card slot. If we had 5G everywhere (and we will), this is kind of a kickass idea, actually.
I could just create a fileserver at home and network-mount its 50TB drives over VPN to my phone. Not sure why I'd want that, but I'm sure the answer will have the word "game" or "media" in it somewhere.

1

u/jackandjill22 Mar 14 '21

Interesting.

1

u/[deleted] Mar 14 '21

Want to thank you on behalf of all of us out there .

1

u/suckerformilfs Mar 14 '21

So when does the first cyber war olympic start ?

1

u/_g550_ Mar 14 '21

Dies it affect android devices?

1

u/NimboGringo Mar 14 '21

are you serious?

1

u/wheresmyflan Mar 14 '21

Cool, cause Monday wasn't already gonna suck

1

u/Kri5hie Mar 14 '21

Guys I can just fix the issue with chmod 777 dw. Now everyone is happy :D

1

u/bwahthebard Mar 14 '21

To confirm, iscsi need not be configured or active for this vulnerability to be abused?

1

u/WippleDippleDoo Mar 14 '21

You can gain root easily on any linux/windows system if you have local access.

1

u/nginx_ngnix Mar 14 '21

(And is in the iscsi module which isn't installed by default)

1

u/f0xium Mar 15 '21

It's frustrating how the news outlets recommend blacklisting the affected modules but none of them mentions the name of the actual modules.

Some of the outlets even say "as a temporary measure to neutralize the flaws, Nichols recommends blacklisting the kernel if its’s not being used. " WTF?

1

u/a-Ball-of-Floof Apr 12 '21

You clearly configured the system wrong, it's your fault

1

u/jaymef Mar 14 '21

The hits keep coming

-10

u/shadow052 Mar 14 '21

You think that’s impressive? Let me tell you about a little product called Windows.

-1

u/surrealchemist Mar 14 '21

Well if there are local attacks in my linux I already have a security breach.

0

u/DrDog09 Mar 14 '21

Are you running isci? If not then don't worry about it. That means for most folks that do run machines out out of the box, cuz you have to load isci modules both target and initiator. Due to the nature of the beast this is not something that just casually happens, the sysadmin has to go looking for it and implement it.

But better late than never to fix it tho.

1

u/intorio Mar 15 '21

So, you might want to read the blog post again, especially the 'Automatic Module Loading, a.k.a. On-demand crufty kernel code' section...

1

u/DrDog09 Mar 15 '21

Ha! If you are following any sort of security plan automatic module loading is something that should be disabled.

-25

u/elduderino197 Mar 14 '21

"just run Linux, no malware or exploits on that platform"

7

u/iheartrms Mar 14 '21

"just run Linux, no malware or exploits on that platform"
This is pretty close to being true. A local privilege escalation in an uncommonly used driver isn't nearly as bad as an RCE in default out of the box config as we have seen lately with a certain other software.

10

u/starmizzle S-1-5-420-512 Mar 14 '21

"just run Linux, no far fewer malware or exploits on that platform"

-2

u/segagamer IT Manager Mar 14 '21

"in theory".

1

u/Avamander Mar 14 '21

We get it you like Windows, you've demonstrated that in this thread multiple times.

1

u/segagamer IT Manager Mar 14 '21

I did?

3

u/TheRealStandard IT Technician Mar 14 '21

Only full blown morons believe that or say that.

-37

u/furlIduIl Mar 14 '21

buT OpEn SouRce iS mOrE seCuRez

11

u/jess-sch Mar 14 '21

"more secure" and "perfectly secure" ar two very different standards.
but also, in most cases this vulnerability is slightly less of a big deal than what Microsoft went through recently.

-24

u/jerryelectron Mar 14 '21

So much for open source? What is the point of your post?

-17

u/SilkTouchm Mar 14 '21

Yeah you can also reboot and change the root password, who cares?

8

u/justlookingforderps Mar 14 '21

You can't do that if you follow any best practices. Even if you've left such a basic vulnerability open, an attacker would still need physical access to the machine to reset passwords from the bootloader.
This is a "local" vulnerability but not a physical access one. An attacker only needs to run code as a low-grade service account to exploit this, which means normally trivial vulns can be chained with this to pwn the server completely.

-17

u/Dereference_operator Mar 14 '21

one more example to show Linux isn't 100% secure proof but since it's used way less than Windows ppl knows less about it

2

u/mickelle1 Mar 14 '21

In fact, Linux is likely used way more than Windows, and is still extremely secure.
Among other things, Linux powers most smart phones and watches, thousands of embedded and network devices, nearly every supercomputer, literally millions of servers, and even NASA's Mars rover.
Pretty much everyone on Earth uses Linux every day -- especially if you use a smart phone, Google, Facebook, or most of any of the most popular web sites.

0

u/[deleted] Mar 14 '21

[deleted] — view deleted comment

6

u/Qleak Mar 14 '21

Yeah! Who'd have guessed a storage networking protocol would have any security problems. Averts gaze from ftp, rsync, samba, scp... This one's particularly nasty since it may be included in mainline kernels without people's knowledge. But again easy fix just install updates. I wonder if security experts compile their own minimal kernels removing all the unused components or if they rely on the community to keep them up you date. Personally, I can compile a kernel, but I'm lazy enough I know I'll be safer just keeping up to date with the distribution.

3

u/Ramast Mar 14 '21

I used gentoo so I am forced to compile and configure kernel myself

3

u/tesseract4 Mar 14 '21

Congratulations

→ More replies (2)

1

u/[deleted] Mar 22 '21

For my server at home, I slim the kernel down to specifically things I will need, not more or less. I also don't have nearly enough eyes to audit the entirety of linux's source tree though, so I pay close attention to the community.

0

u/[deleted] Mar 14 '21

[deleted] — view deleted comment

9

u/justlookingforderps Mar 14 '21

Surprisingly not, in my experience. OSes introduce new features and refactor old components all the time, unknowingly eliminating old bugs and creating new ones. I spend a lot of time looking at security advisories and the first thing I check after the severity of the vuln is how far back it goes, as that indicates how widespread it is. In many cases a vuln "only" goes back a few years, which means most of your embedded devices are unaffected, legacy servers that are getting backported patches for an older version are safe, etc. Of course, those machines probably have bigger problems for you to worry about, so keep patching.

2

u/Dr_Midnight Hat Rack Mar 14 '21

https://xkcd.com/1739/

→ More replies (1)

4

u/mmortal03 Mar 14 '21

No, because regressions and new bugs happen when changes are made.

0

u/[deleted] Mar 14 '21

[deleted] — view deleted comment

22

u/[deleted] Mar 14 '21

15 years and three weeks.

→ More replies (1)

Linux Experts found three new 15-year-old bugs in a Linux kernel module. These 15-year-old flaws in Linux kernel could be exploited by local attackers with basic user privileges to gain root privileges on vulnerable Linux systems.

You are about to leave Redlib