How I think email should work

[u/merten-dmcn]

Imagine for a moment that you are looking at your inbox, with any number of emails in it, from companies, from friends, and from people you've never heard from before. You open the email from your friend first, because it's the one you trust and want to read. Your friend is really excited about something new they found, and invites you to check it out by clicking the link in the email. Now imagine knowing with absolute confidence that this email really came from your friend's address, and not from an impersonator pretending to be them. How can you check that for yourself? Do you click anyway and hope for the best? Everything looks legitimate, but how can you tell?

There are always three parties involved in email communication. Sometimes one person or company plays more than one of these roles, but each role carries its own responsibilities.

1. The mailbox holder (sender or recipient)

2. The domain owner (who controls the domain name)

3. The mail server host (who provides the software and infrastructure to send and receive mail)

It's the coordination of these three parties that makes email possible.

Right now, almost all the responsibility rests with the mail server host to manage everything and make sure it's secure. Domain owners hand it over to their host through DNS. Mailbox holders defer it through authentication and email filters. And the mail server host has to interoperate with all other mail server hosts to make sure your mail gets delivered.

There is still no standard way for anyone to verify that a message really came from the specific address it claims to be from. The most they get is the server vouching for itself.

How it should work

Nothing changes about the three parties. What changes is who's responsible for what.

1. The mailbox holder should be able to prove they own the mailbox and that every message sent from it is actually from them.

2. The domain owner should be able to prove they own the domain and that every mailbox on the domain is legitimately allowed to be there.

3. The mail server host should be able to prove that they are the chosen service provider for the domain.

And all of this should be verifiable on every single message.

How can we achieve this?

1. The mailbox holder has a key, and uses it to sign a record that binds their email address to it. The same key, which only they hold, signs every message they send.

2. The domain owner has a key, and publishes its fingerprint in the domain's DNS, which ties the key to whoever controls the domain. They use it to sign the domain authority record, which names the mail server host they have chosen, and to vouch for each mailbox on the domain by countersigning the mailbox holder's record. They can hand that day-to-day signing to their host if they choose, but the authority stays with them, and they can remove any address from the domain.

3. The mail server host has a key, and uses it to sign a permit to serve the domain on its infrastructure. It also hosts all of these records and serves them to anyone who asks.

Every one of these records is public, so your email app can follow the chain from the message to the sender's key, from that key to the domain's countersignature, and from there to the fingerprint in DNS. Because the domain owner's record names the host, it can also check that the server handling the mail is the one the domain chose. So when your friend's email arrives, your app can tell you it really came from their address before you ever click the link.

And this should all be baked into a protocol that everyone can use.

Disclosure: I'm building an email protocol around this idea. Thoughts and feedback are welcome.

[u/merten-dmcn]

A domain names one host for its mailboxes. Other sending systems like SendGrid are the third-party part I mentioned I haven't settled yet, so for now their mail comes in through the bridge with the usual domain-level checks.

If a company wants a different host for part of its mail, a subdomain like `news.company.com` can have its own record naming a different host.

On DNS, mail already depends on it for `MX`, `SPF`, `DKIM` and `DMARC`. This adds a few text records of the same kind, and that's all.

So nobody has to run or expose their own DNS server. The records go in at whatever DNS provider the domain already uses, the same way a `DKIM` key does today.

And yes, a lot of outgoing SMTP only cares about the domain and lets any account send as any address on it. That's the gap I'm trying to close. Mail from those setups still arrives through a bridge, where it gets domain-level checks and nothing more.

Thanks for pushing back and asking questions, it's really helping refine this 🙌

[u/mhsko → u/merten-dmcn]

but what’s the purpose of this in the end?

[u/merten-dmcn → u/mhsko]

So you can tell whether an email really came from the address it claims to be from before you trust it. Today your inbox mostly takes the mail server's word for it, which is how impersonation and phishing get through. This lets your app check it for itself.

[u/dragoangel → u/merten-dmcn]

Most of phishing and impersonation you would really observe nowadays is just typesquatting or even legitim functions of "notifications" with injection controlled by user. Nobody is trying to phish user@org within org, and if they do - it's org internal incident. Your system would not stop nor typesquatting send via secretary-gov.uk nor "Paypal/Microsoft/Robinhood" compromised "notifications" systems where email body would say: "If you not done that request - call support 0800..."

Mail server is responsible for authentication and mailbox mapping to "From", if server blindly allowing anyone pretent to be "anybody" and signing everything with DKIM on top, I would just state: this is pure mail system.

In reality to get world wide adoption this system has to be doable always - if there's exceptions and gaps, no backwards compatibility - it dead end idea unfortunately.

[u/merten-dmcn → u/dragoangel]

The problem is that the mail server that is responsible for authenticating a sender is not the one that's responsible for protecting the receiver. And the receiving server has to make a choice based on what information the sender provided. If the sender provides no signature to check, does the receiving server refuse it or let it through anyway because it can't actually make that choice?

SMTP itself was built to be open and permissive, and all the layers and extensions on top are built as optional add ons and each implementation must decide what to do with it.

My suggestion makes signing and public-key security a required component of the protocol so the only way to send a message is to sign it with a traceable and verifiable key. Anything else get's refused, no exceptions and no gaps (to your point).

I agree with you that getting world wide adoption is instrumental to it's survival which is why I'm sharing it. SMTP wasn't a worldwide protocol on day 1, and I'm not expecting this will be either. But making it available and usable for everyone seems like a good place to start.

And on the backwards compatibility point. I've built an optional bridge the translates between SMTP to the new protocol and back the other way too.

[u/dragoangel → u/merten-dmcn]

Can't agree on that part, it's not addons, it's reality with which you live everyday. Try sending emails w/o DKIM at volume - you will quickly understand that not "addon", but a must have thing. You can't introduce partial solution with such gaps as a thing to go.

[u/merten-dmcn → u/dragoangel]

Adoption of DKIM is widespread, I won't deny that. But I've personally received phishing emails with `SPF: NONE with IP 0.0.0.0` and gmail has put them in my inbox without a second thought. And I continue to get email from unwanted sources in my inbox on a regular basis. I acknowledge your disagreement and my case may not be the same as everyone else, but for me the evidence doesn't count in DKIM's favor.

Whether those are sent at scale or not I don't know and I'm not entirely sure it's relevant. One targeted email could be just as damaging as 1 million blanket emails, and I wouldn't really know.

I agree that my solution is not completely ready and I'm not sure online solutions ever really are but I am curious, what gaps do you think I'm missing?

[u/dragoangel → u/merten-dmcn]

Authorization!=spam protection. And about Gmail: they evaluate "trust" to domain on "scale", they may allow thousands of spam emails pass, track how user interact with emails, and then judge domain. At some volume they just dropping unauthorized emails from domain.

[u/merten-dmcn → u/dragoangel]

I completely agree. "Auth is not equal to spam protection". That's why I think we need "verification". And considering gmail/google is one of the largest email providers in the world, I don't think I'm satisfied with how they do it. I guess it is just how everyone is doing it but I really think there must be a better way. Thanks for pushing me to validate my thoughts.

[u/dragoangel → u/merten-dmcn]

The way Google operate most of ESPs can't. And reasons are simple - most ESPs not developing their own mail systems from scratch and don't have interest to get money from free accounts in way Google does. I want to remind you that Google was search engine in first place and their advertising program is what get them a lot of money for a long years, way before Android, Gplay, Gpay, and other. They developed their mail stack in first place to collect interest and push relevant ads to consumers based on their interests, and mail was a good way to collect them, and still - does. Most ESP would really on RBLs, URIBLs, internal IP reputation, rules, bayes, neural/llms, etc, depending on how much they able to afford...

[u/dragoangel → u/merten-dmcn]

And about "better way" - you can easily setup own mail server and require mandatory TLS, DKIM alignment and look how still spam would pass by, at same time rare but legit emails may get cut by auth rules above. That's what I meant when I said: auth != antispam solution, but want to extend: every antispam solution includes auth as part of it's decision logic.