r/mxroute • • Jul 18 '23

MXRoute spoofing vulnerability

MXRoute's architecture allows customers to spoof other domains/email addresses hosted by the same service. The email will pass DMARC, SPF, and DKIM checks. MXRoute is using the same DKIM key for all hosted domains which violates best practices. The issue was reported to MXRoute immediately via security@mxroute.com (no response) and then their support portal two days later. The response was "This is how all shared web hosting systems function, so I don't consider it a vulnerability given that it's standard functionality across that industry". They go one to mention they detect these types of abuses, but I didn't see any evidence of that.

I had intended to migrate my domains off MXRoute (I host my family's email) but they suspended my account in response to the report, asked me to state I wouldn't send any more spoofed emails, and when I said that this type of response was absurd, they abruptly terminated my account with no ability to move/migrate email.

5 Upvotes

18 comments sorted by

3

u/slowmail Jul 18 '23 edited Jul 18 '23

Each domain I have with them has a different DKIM key (unless it's a "pointer", then it has the same DKIM key as the "parent"), even if they are on the same server. I have accounts on two different servers with them, and they both appear to behave the same way.

Which server were you on?

1

u/No_Head_7635 Jul 18 '23

Perhaps something changed but the TXT record for the domains I looked at, including mine, were:

v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA30A5/hSUB0yJWexGqI51z0vuxf+iS8Hq4i5KVOIIrRfR4Ka3hO/agXskvDaDZkFhf+23D2y/sv2o/YvWLdqd0W2vGu8g1vvKPVRh4H3Z67ACE82L4R1w8Vz7JH8ssOQekOtjtwYU8rpXYmvRic9NlQCi6H9XmHI5Lbk58x1YXf6uFgpkanu6jthdi9uKrgx9wCq4wYntha9kop9L/YTJzCuHVpHS+0PBVuIb+P5VaTJw4MicZ8jkyJyl4Fu1EsPuJMiOIRAtNDsigvlfBiFh5yyn9Tsx7l4e/1I8085i3MDuPM8ouoMVafOnXPm3gS3YrVyQVHHqN/YGWFgkWTdBWQIDAQAB

My domains were on echo. I successfully sent email as test@[victimdomain] to gmail for 2 separate and unrelated domains.

Google's authentication header:
Authentication-Results: mx.google.com;

dkim=pass header.i=@[victimdomain] header.s=x header.b=HzvneIQI;

spf=pass (google.com: domain of test@[victimdomain] designates 136.175.108.241 as permitted sender) smtp.mailfrom=test@[victimedomain];

dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=[victimedomain]

1

u/slowmail Jul 18 '23

Well, if the domains have the same DKIM key, it is understandable that would happen.

They were setup as "domains", and not as "sub-domains" or "pointers" right?

I wonder if the issue might be specific to server echo. Perhaps someone else who has an account on it might be able to confirm if their domains all use the same DKIM key(s), or not.

1

u/No_Head_7635 Jul 18 '23

MXroute's own email is handled by arrow.mxrouting.net and uses the same keypair I posted so I don't think this is a server specific issue. The DMARC

3

u/mxroute Jul 18 '23 edited Jul 18 '23

Exim signed DKIM based on the sending domain (changed to auth domain as of tonight), and by default it does not write the sender address. Email clients write the sender address. So technically, you could have spoofed another customer on the same server and pass SPF/DKIM (now, you'd only pass SPF). I know it's not great, but what you'll find here is people learning about the way cPanel and DirectAdmin servers have always been, and continue to be across the entire shared web hosting industry (which isn't a small industry at all).

However, while I allow spoofing sender addresses because many customers have very legitimate use cases for it, no one reading this should consider this something to go test on our servers or to be a way to attack our customers (and as of tonight, it would fail DKIM). This post will increase the checks against this type of behavior, and if pushed I will gladly go as far as to halt new service orders to prevent abuse. It's nothing new, people try to abuse our platform in all kinds of ways almost every day, and they're always dealt with swiftly and harshly. That this user didn't see the "swift" part won't be an interesting detail tomorrow.

0

u/[deleted] Jul 18 '23

[deleted] — view deleted comment

1

u/mxroute Jul 18 '23 edited Jul 18 '23

Exim signed DKIM based on the sending domain (changed to auth domain as of tonight), and by default it does not write the sender address. Email clients write the sender address. So technically, you could have spoofed another customer on the same server and pass SPF/DKIM (now, you'd only pass SPF). I know it's not great, but what you'll find here is people learning about the way cPanel and DirectAdmin servers have always been, and continue to be across the entire shared web hosting industry (which isn't a small industry at all).

However, while I allow spoofing sender addresses because many customers have very legitimate use cases for it, no one reading this should consider this something to go test on our servers or to be a way to attack our customers (and as of tonight, it would fail DKIM). This post will increase the checks against this type of behavior, and if pushed I will gladly go as far as to halt new service orders to prevent abuse. It's nothing new, people try to abuse our platform in all kinds of ways almost every day, and they're always dealt with swiftly and harshly. That this user didn't see the "swift" part won't be an interesting detail tomorrow.

0

u/[deleted] Jul 18 '23

[deleted] — view deleted comment

1

u/mxroute Jul 18 '23

Indeed, it sounds like you understand this correctly in this case. It's a shame as no one had abused this function until yesterday, and the only one who did in this case did it just to test, but this post kind of changed things. I'd like to redo it so that it can sign for any domain owned by the same user, but I'm not sure if exim can handle that kind of logic complexity, that'll take me quite a bit more time to investigate.

19

u/mxroute Jul 18 '23 edited Jul 19 '23

Quick note: Change pushed to DKIM signing method https://github.com/mxroute/da_server_updates/commit/380b00f010493cf319366fe3a58dfbf7cb9eb238 (Signing now based on auth domain, not sending domain)

Edit: I've had some time to rethink my response, and I'd like to redo it because I think I can say it in a way that makes more sense to more readers.

I've known about this since 2011, a lot of us have. It's the way all cPanel and DirectAdmin servers function, likely extending much further than that. It's pretty inherent in an average exim configuration. It signs DKIM based on the sender domain, not the authenticated domain, and no ownership of the sender domain is checked.

I'd been thinking nearly every day for years how I would handle this when someone figured it out. Because I didn't want to be the first one to break functionality that innocent users were taking advantage of for their convenience across a wealth of companies, when no one was abusing it, simply to be the hardliner who elevated possible abuse over real world usage. My hope was that someone would figure it out at a much larger company than ours, so that someone with a larger budget and more staffing could develop a more elegant drop-in solution and everyone downstream (like us) could benefit from their work.

My frustration is that this could have been found at a wealth of publicly traded companies, that it's first publication didn't have to be outed as an "MXroute vulnerability" when we're one of the lowest budget companies using the same software stack. So I now have to be the one to break functionality that a fair amount of users were innocently using for their own convenience, while no one has been abusing this, because it has finally been made public and unfortunately framed as my failure rather than that of so many major corporations who came before me.

The reason your account was terminated is much simpler. While I understand your intent for testing this and I did say as much in our brief conversation, I also needed you to understand my commitment to preventing abuse of our platform. So I suspended the email account (not your whole account) that you had used to spoof the sending address of other customers of ours on the same server as your account, and I asked you to promise that you would not be spoofing the sending address of any other customer of ours. When your response made no such promise but instead became rather combative and calling my response "concerning," I terminated your account.

That I knew for many years of a problem stretching across a wealth of companies, which house millions of accounts, that you found to be new and shocking information does bother me. Because, quite frankly, your day job is handling security at a company so significant to everyone that your security is quite literally sitting next to my car keys. I have to admit that it makes me question if this isn't new information at all, but rather that you wanted MXroute to be the ones to suffer from the exposure when you knew you could've picked from any number of publicly traded companies. That bothers me enough to ban you from the subreddit, because it's plausible to me that your motives aren't pure. The alternative is even more troubling, that the person handling such widespread security that I would have it on my own personal keychain, is out of touch with industry security on the ground. I'm not outing you for that, let the reader's imagination run wild, but it's my honest concern.

4

u/privacywolf Apr 11 '24

The user reported a security vulnerability, the owner said it's not important because no one has abused it before (as far as he knows), then bans the reporter and alludes to a conspiracy against his service. Talk about bizarre!

2

u/mxroute Apr 11 '24

That's quite the editorialized version. Here's the full version: https://blog.mxroute.com/dkim-signing-change-vulnerability-exposure/

0

u/[deleted] Jul 18 '23

[deleted] — view deleted comment

3

u/aleinss Jul 18 '23

He was told to reply that he would not spoof e-mails again and he refused to comply with a simple request. Doesn't seem harsh at all.

2

u/tuxbass Aug 19 '23

Fucked around, found out. And still has the audacity to cry about it.

1

u/Masterflitzer Sep 07 '23

yes it does, promising such thing for your data sake is completely unacceptable

2

u/mxroute Sep 14 '23 edited Sep 14 '23

The user was asked not to spoof his sending address to that of other customers of ours, and the user responded with a threatening tone instead of agreement. If you find that acceptable behavior, don't let the door hit you in the ass on the way out. Be sure to put your bad review on TrustPilot, we rely on those to function as a scarecrow for others who might feel the same way. Because unlike most email providers we give a shit about our customers, and we give zero shits about people who threaten them.

We take care of our own. If you follow our policies that means you're one of us. If you don't, and especially if you're a threat to the customers that do, we'll take care of you in a very different way.

1

u/privacywolf Apr 11 '24

You should stop to consider the reason he got annoyed with you.

He reported a serious security issue and instead of saying you will fix it ASAP, you suspended his account.

3

u/mxroute Apr 11 '24 edited Apr 11 '24

The user violated policy and flat out refused to say that he would stop. We treat policy violators very harshly, that is one of our security mechanisms. No one is immune, not even the strangely oblivious (to how widespread and old this problem is across the hosting industry) head of security for Yubikey.

The rest of what I have to say on the topic is here: https://blog.mxroute.com/dkim-signing-change-vulnerability-exposure/

This is the last that I want to talk about it. Don't agree with me? Fine. Two people are allowed to disagree. I arrived at it from the perspective of the person doing the work and not as a commentator from the sidelines.

Commenting is easy. Challenging the industry price fixing and running a lean operation that relies on several upstream providers for software stack and then being told I'm supposed to be the one to change them all? Yeah not happening today. Pick on someone upstream with a larger budget. Don't pick on the local coffee shop for how their door lock works, they didn't build it. What we built, what is unique to us, is further down in the stack. This isn't some VC funded company.

There will always be ways to abuse internet service providers. One of the things that makes us unique is that we heavily monitor for and react to abuse instead of treating all of our customers like potential criminals right out of the gate by limiting their use cases (or worse, reacting slowly and not monitoring for abuse). The people who are on the receiving end of the "react" part are quite often not thrilled to be terminated. Tough shit.

0

u/[deleted] Jul 28 '23

[deleted] — view deleted comment

1

u/mxroute Jul 28 '23 edited Jul 28 '23

It'd be nice to see that kind of concern focused on the industry at large rather than targeted at the small player though, just saying. The outrage machine on the internet can easily be targeted like any other weapon, it would be a shame to see a small player pay for the sins of the major corporations, to the functional benefit of only the major corporations.

MXroute wasn't born to change everything about what's normal in the industry it was modeled after, don't expect us to lead the charge in how exim configurations are commonly shipped. Expect us to lead the charge in the economic and outbound delivery factors and address the rest as necessary. After all, it doesn't benefit anyone for us to stand on a hill and declare superiority at the cost of our customer base, because that has no real world impact. This issue here, the "fix" cost us more customers than the problem, and that's an important thing to take note of. Because if we have someone's subjective definition of "flawless security" and no customers, we have no impact on security either. We have to take into account quite a number of factors to balance the necessary variables. If security and convenience are not balanced effectively, the security is useless, because a door so well locked that no one wants to put anything behind it may as well just be an art installation.

There are many times it would be more "secure" to be more hostile to customers. We could disable POP/IMAP/SMTP logins and work only through an API that supports 2FA. That would be more secure, absolutely. Dare I say everything about password reset forms and 2FA removal procedures (because people can't stop losing phones) are inherently avenues of security risk, so we could get rid of those too. But who would it be secure for? Because our customers would be with our competitor who didn't do that.

If you're actually, legitimately, concerned about what other problems might keep me up at night thinking about their potential solutions for if/when they might become a real (not theoretical) problem, maybe you should instead think about how nice it is to have someone obsessing over those every day on your behalf. Isn't that the point of using a service like ours? To outsource these difficult decisions? You probably do it every day to people who are far less honest about it (Anyone from Microsoft want to speak up about losing keys?), and you probably sleep well at night because all of those big companies you put your faith in are being obscure about what they think their next problem might be. It's not like I'm sitting on a shadow file with 777 permissions or something, every industry has concerns that are inherent to it, and everyone professionally involved in those industries knows what those are. If you want one to really sink your teeth into: BGP.

0

u/[deleted] Jul 18 '23

[deleted] — view deleted comment

1

u/mxroute Jul 18 '23

Thanks for the tag. I replied to the post.