Re: [gentoo-user] tips on running a mail server in a cheap vps provider run but not-so-trusty admins?

[Grant Taylor]

On 8/21/20 6:37 AM, Rich Freeman wrote:
> This stuff can be interesting to discuss, but email is SO entrenched
> that I don't see any of this changing because of all the legacy issues.
> You would need to offer something that is both easier and better to
> use to such a degree that people are willing to change.

"something that is /both/ *easier* and *better*".

That's a VERY tall bar to clear.

> 1. Modernizing the actual data exchange using http/etc.
>
> I don't think you'll get much argument that SMTP isn't the way
> somebody would do things if they were designing everything from
> scratch.

SMTP may not be the best, but I do think that it has some merits. 
Merits that the previously mentioned HTTP/2 alternative misses.

Remember, SMTP is a protocol, or a set of rules, for how two email 
servers are supposed to exchange email.  These rules include the following:

1)  Proper server greeting, used to identify the features that can be 
used.  (HELO for SMTP and EHLO for ESMTP.)
2)  Identifying the sender.
3)  Identifying the recipient(s).
4)  Sending the message payload.

Each of these discrete steps is a bi-directional communication. 
Communication that can alter all subsequent aspects of a given SMTP 
exchange.

The server could be an older server that only speaks SMTP and as such 
doesn't support STARTTLS.  Thus causing the client to /not/ try to use 
STARTTLS in accordance with current best practices.  Other ESMTP 
features come into play here like message size, 8-bit, notification 
features, etc.

The receiving server has the ability to assess the sending server's 
behavior at each point to use that as an indication to know if the 
sender is a legitimate email server or something more nefarious and 
undesirable.  --  There is a surprising amount of email hygiene based on 
sending server's behavior.

The receiving server may carte blanch reject messages from specific 
senders / sending domains and / or specific recipients, particularly 
non-existent recipients.

SMTP has low overhead in that the message body is not transferred from 
the sending server to the receiving server if any part of steps 1-3 
aren't satisfactory to the receiving server.

> I'm not an API expert but I imagine that just about any of the modern
> alternatives would be an improvement.

Maybe.  Maybe not.

What is the actual problem with SMTP as it is?

What /need/ to be improved?  What could benefit from improvement even if 
it's not /needed/?

> http would be a pretty likely protocol to handle the transportation,

Why HTTP?

Did you mean to imply HTTPS?

Why add an additional protocol to the stack?

TCP / SMTP is two layers.

TCP / HTTP / $Email-protocol-de-jure is three layers.

UDP / HTTP / $Email-protocol-de-jusre is three layers.

Why introduce an additional layer?

Why introduce an additional dependency in the application stack?

Why co-mingle email with other non-email traffic?  --  Or are you 
wanting to run HTTP(S) on TCP port 25?

> likely with something like JSON/xml/etc carrying the payload.

Why add an additional encapsulation on top of the additional layer?

What does JSON / XML / etc. on top of HTTP(S) provide that SMTP doesn't 
provide?

What is the gain?

Is the gain worth the cost of doing so?

> You might want to tweak things slightly so that recipient validity can
> be tested prior to transferring data.

Definitely.  (See above for more details as to why.)

But now you are doing multiple exchanges.

If we extrapolate this out to also include sender validation, and 
probably hello messages, you are back to four or more exchanges.

How is this better than SMTP on TCP port 25?  What is the benefit?  What 
does it cost to get this benefit?

> A mail server doesn't want to accept a 20MB encrypted payload if it can
> bounce the whole message based on the recipient or a non-authenticated
> sender or whatever.

Which is one of the many reasons why there are multiple exchanges back 
and forth.

> However, in principle there are plenty of modern ways to build a
> transport protocol that would be an improvement on SMTP ...

Please elaborate.

Please include what network layer (3) protocol(s) you want to use and why.

Please include what application layer (7) protocol(s) you want to use 
and why.

Please include any encoding / encapsulation you want to use and why.

> ... use more standard libraries/etc.

There are many libraries to interface with SMTP.

Also, handing a message off to an SMTP server alleviates the application 
from having to deal with failed deliveries, queuing, and retrying.

Why add that complexity into each and every application?  Especially if 
you don't have to?

Note:  Pulling it in via a library is still indirectly adding it to each 
and every application.

How is SMTP /not/ standard?

Also:  https://xkcd.com/927/  --  Standards

> I think this is actually the part of your proposal that might be
> more likely to take off.  You could have a new type of MX field in
> DNS ...

So yet more complexity in addition to the extra layers of the stack.

> ... if it is set then it defines servers that will use the new protocol,

It defines servers that are purportedly /capable/ of using the new protocol.

Advertising a capability is entirely different than using an advertised 
capability.

> and you could have backup SMTP servers for legacy transition.

We've been transitioning from SMTP to ESMTP for about 28 years.  How 
long do you think it will be before we finish transitioning away from 
stock SMTP?

Further, how long do you think it will be before we finish transitioning 
to your new proposed standard?  I'm guessing at least another 30 years 
to finish it.

So how long will we have three email protocols; SMTP, ESMTP, and 
$Email-protocol-de-jure?

> You could change the transport side of things without redefining how
> email works, so the same messages are interoperable with both systems.

So now you are proposing putting the SMTP protocol on top of a new 
transport instead of TCP.

Old:  TCP / SMTP is two layers

New:  TCP / HTTP(S) / SMTP is three layers
New:  UDP / HTTP(S) / SMTP is three layers

What exactly does moving SMTP from TCP to HTTP(S) provide?  What is the 
benefit for the added complexity and dependencies?

> On the flip side, while I think this is an easier change to make,
> I don't think it necessarily offers huge improvements.

So why bother doing it at all?

> SMTP still  gets the job done,

Yes.  SMTP does it's job sufficiently well.  Is it the best method? 
Doubtful.  Is it well supported and widely used?  Yes on both accounts.

Note:  The same can be said about Ethernet too.  Yet here we are 30 
years later still using it and enhancing it to be 40,000 times faster 
than it originally was. (400,000,000,000 bps vs 10,000,000 bps)

> and the new system wouldn't really be a major improvement from a
> sysadmin standpoint, so I don't see everybody running out to change.

If there's not a significant* change, why should we bother changing?

Where significant is enough to overcome the work effort and pain to make 
the change.

> Just changing the transport protocols doesn't provide any anti-spam/etc
> benefits.

Nope.  So no benefit and additional complexity / overhead.  Sounds like 
a net loss to me.

> SMTP already supports TLS so it isn't any more secure.

Just because (START)TLS is provided doesn't mean that it's used.

There may be some benefit to using a transport that has implicit 
(START)TLS encryption.

We could easily revisit SMTPS, which does have implicit TLS encryption 
on TCP port 465.

SMTPS is a well establish, but seldom used, way for clients to send 
email to their outgoing mail server.  Perhaps we could re-use this as a 
new server-to-server SMTP transport.  Slightly modifying and re-using 
prior art is almost always better received than something completely new.

There is also the possibility of working on augmenting SMTP, or more 
likely the metadata around it, to make existing SMTP servers to 
/require/ STARTTLS when it is known that both the sending and receiving 
servers support it.

This latter idea is already somewhat being done by many email 
administrators.  Admittedly it's sub-optimal and likely manual.  But 
that sounds like a simpler change.  Perhaps publishing a flag in DNS 
could indicate that email from my servers should be STARTTLS encrypted 
and you should require it on the receiving server (presuming you support 
it).

> No doubt ...

I have serious doubt.

> ... if we were starting from scratch this is how it would be done, but
> it isn't enough of a reason to change.

DING DING DING!!!

> That said, if there were even modest benefits for large mail providers
> I could see it taking off simply because it could leverage a lot of
> existing libraries and standards, and could have legacy compatibility.

But we already have MANY libraries and MANY standards.  What good does 
yet another one provide?

Stop focusing on HTTP(S) <any version thereof> for a few minutes.

If you were to create a new protocol from the ground up, why would you 
introduce additional layers into the application stack — which represent 
additional dependencies and complications — if you didn't have to?

Not everything is a web server.  Not everything can benefit from being 
forced through a web server.

> 2. Changing the model around email moving to something more
> PKI-based/etc and redefining what an email address is.
>
> This is where you could have revolutionary improvements for
> privacy/spam/etc.

What does changing email addresses (user part and / or domain part) 
provide functionality that can't be done — as in is not at all possible 
to do — with current email addresses?

> However, it completely breaks everything that exists with email today,
> so it is a hard change.

Changing email addresses doesn't break email as it exists today.  /SMTP/ 
/doesn't/ /care/ what the addresses are.

/Humans/ /using/ /email/ (thus indirectly SMTP) /do/ /care/ what the 
addresses are.

> Most of the benefits only accrue if you adopt PKI as well, which is
> a problem that is hard to scale out to the masses.

PKI is relatively easy to scale.  (For a given value.)  Hence the 
thriving CA industry.

Doing so in a trust worthy manner is the problem.  Especially if you 
want to avoid a single point of control / failure / GEO political / etc.

> If somebody comes up with a way to make PKI take off in a real way,
> then I could see something like this taking off eventually.  Otherwise,
> this stuff is only going to be of interest to the sorts who use gpg
> to sign their email, and nobody else.

The biggest issue I see with PKI is /bootstraping/ trust.

If we throw all the trusted CAs out the window, how do you /bootstrap/ 
encrypted communications?  -  My personal opinion is DNS.

We already have a very well understood, globally distributed, highly 
redundant database with unique keys.

We have methods to authenticate it; DNSSEC.

We have ways to hid it's use; DNSCrypt, DNS-over-TLS (DoT), 
DNS-over-HTTPS (DoH), DNS-over-53-over-IPsec (transport mode), and VPNs 
(IPsec (tunnel mode), OpenVPN, WireGuard, et al.

So, why not put recipient's public keys in this phenomenal 
infrastructure that we already have in place?

Oh, we did.  There are already ways to publish S/MIME keys in DNS.  I'm 
fairly certain that there are ways to publish PGP keys in DNS too. 
Non-conflicting ways at that.

We also have ways to publish the public key of a host's public / private 
key pair.

The sad thing is that most people don't know that these well defined 
methodologies exist and of those that do, few use them.

The long and short is that there are some relatively simple things that 
we can do to augment the existing SMTP infrastructure to make it better. 
  We do not /need/ to do a wholesale replacement and incur all the 
complications ~> technical debt that goes along with such a replacement.




-- 
Grant. . . .
unix || die