This page documents the configuration of the mail server. There is no user-facing documentation yet.

Running a mail server can be surprisingly hard, considering how long the protocols have been around. However, I believe it is something that can be managed by any moderately experience administrator, and in fact could be better automated if we could figure out a simple set of standard configurations. Hopefully, this page should cover the latest good configuration for small servers.

  1. Basic mail server configuration
  2. DNS
  3. Postfix
  4. Postfix opportunistic TLS
  5. Postfix SASL configuration
    1. Dovecot
    2. Postfix server
    3. Testing
    4. Client configuration
    5. References
  6. Delievery and retrieval over SSH
  7. Spam filtering
    1. Todo
  8. Dovecot and mail filters
    1. Todo
  9. Webmail

DNS

One of the trickiest parts of configuring email is to setup DNS correctly. You must have an A record that points to your server's IP address that is always up to date, and a reverse PTR record that points to the same name. Otherwise a lot of servers will bounce your outgoing emails.

Furthermore, some providers block certain IP blocks as being "consumer" addresses that are never supposed to send email. There is little you can do against this but hope for the best or find a provider with "clean addresses".

It is also possible that the address you are given by your provider is blocked because it sent spam in the past: in this case your only hope is to get the address unblocked or have your provider give you another IP address.

Postfix

I am using the Postfix server. I find it easier to configure than the other common mail server, Exim, which I never bothered learning.

Debian does a good job at configuring Postfix for you, when you install the package. It does ask a bit too many questions for my taste, but it does help with the grunt of the work. Here are, some pointers to the answers you want to give:

There are literally hundreds of configuration settings available in Postfix, the best is to the excellent Postfix documentation for further guides. Some of them are simplified below.

Postfix opportunistic TLS

This configures Postfix to offer a X509 certificate on inbound and outbound SMTP connections but also accept unencrypted connections as a fallback, in main.cf:

# configure certificates
smtpd_tls_cert_file = /etc/letsencrypt/live/anarc.at/fullchain.pem
smtpd_tls_key_file = /etc/letsencrypt/live/anarc.at/privkey.pem
smtp_tls_cert_file = ${smtpd_tls_cert_file}
smtp_tls_key_file = ${smtpd_tls_key_file}
# opportunistic encryption
smtpd_tls_security_level = may
smtp_tls_security_level = may

See the [TLS_README][] and [smtp_tls_security_level][] for more information.

This patch is required to disable TLS when a [content_filter][] is configured (see [FILTER_README][] and below).

diff --git a/postfix/master.cf b/postfix/master.cf
index b0ed875..6329c70 100644
--- a/postfix/master.cf
+++ b/postfix/master.cf
@@ -12,7 +12,9 @@
 smtp      inet  n       -       -       -       -       smtpd
         -o content_filter=smtp:127.0.0.1:10025
         -o myhostname=mx.anarc.at
+        -o smtp_tls_security_level=none
 localhost:10026        inet    n       -       n       -       10      smtpd
+        -o smtpd_tls_security_level=none
         -o content_filter=
         -o myhostname=delivery.anarc.at
 submission inet n       -       -       -       -       smtpd

Without this patch, local delivery would hang during my tests.

Postfix SASL configuration

This allows users to submit mails using their regular IMAP credentials.

Dovecot

We don't want to configure a new authentication system for Postfix, so we delegate to the already existing one, which is Dovecot.

In the default configuration, the following patch is required to Dovecot to enable a control socket exposed to Postfix:

diff --git a/dovecot/conf.d/10-auth.conf b/dovecot/conf.d/10-auth.conf
index 1c59eb4..187b262 100644
--- a/dovecot/conf.d/10-auth.conf
+++ b/dovecot/conf.d/10-auth.conf
@@ -97,7 +97,7 @@
 #   plain login digest-md5 cram-md5 ntlm rpa apop anonymous gssapi otp skey
 #   gss-spnego
 # NOTE: See also disable_plaintext_auth setting.
-auth_mechanisms = plain
+auth_mechanisms = plain login

 ##
 ## Password and user databases
diff --git a/dovecot/conf.d/10-master.conf b/dovecot/conf.d/10-master.conf
index e3d6260..5068100 100644
--- a/dovecot/conf.d/10-master.conf
+++ b/dovecot/conf.d/10-master.conf
@@ -93,9 +93,11 @@ service auth {
   }

   # Postfix smtp-auth
-  #unix_listener /var/spool/postfix/private/auth {
-  #  mode = 0666
-  #}
+  unix_listener /var/spool/postfix/private/auth {
+    mode = 0660
+    user = postfix
+    group = postfix
+  }

   # Auth process is run as this user.
   #user = $default_internal_user

Then notify Dovecot of the change:

service dovecot reload

Postfix server

This enables SASL authentication with Dovecot. It ensures passwords are only sent over a secure channel and censors the original IP address.

--- a/postfix/main.cf
+++ b/postfix/main.cf
@@ -46,8 +46,9 @@ home_mailbox = Maildir/
 #mailbox_command = /usr/bin/procmail -a "$EXTENSION"
 smtpd_recipient_restrictions = reject_unlisted_recipient,
                                permit_mynetworks,
+                               permit_sasl_authenticated,
                                reject_non_fqdn_recipient,
                                reject_unauth_destination,
                                check_policy_service inet:127.0.0.1:10023,
                                reject_rbl_client zen.spamhaus.org

In other words, make sure that [permit_sasl_authenticated][] is added to [smtpd_recipient_restrictions][].

Then you need to hook Dovecot SASL authentication into Postfix and make sure it is not offered in cleartext:

postconf -e smtpd_sasl_type=dovecot
postconf -e smtpd_sasl_path=private/auth
postconf -e smtpd_tls_auth_only=yes

Then enable the submission port (587) in master.cf:

diff --git a/postfix/master.cf b/postfix/master.cf
index f4f096d..b0ed875 100644
--- a/postfix/master.cf
+++ b/postfix/master.cf
@@ -15,11 +15,11 @@ smtp      inet  n       -       -       -       -       smtpd
 localhost:10026        inet    n       -       n       -       10      smtpd
         -o content_filter=
         -o myhostname=delivery.anarc.at
-#submission inet n       -       -       -       -       smtpd
-#  -o smtpd_tls_security_level=encrypt
-#  -o smtpd_sasl_auth_enable=yes
-#  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
-#  -o milter_macro_daemon_name=ORIGINATING
+submission inet n       -       -       -       -       smtpd
+  -o smtpd_tls_security_level=encrypt
+  -o smtpd_sasl_auth_enable=yes
+  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
+  -o milter_macro_daemon_name=ORIGINATING
 smtps     inet  n       -       -       -       -       smtpd
   -o smtpd_tls_wrappermode=yes
 #  -o smtpd_sasl_auth_enable=yes

As an extra, you can protect your users' privacy by hiding their IP address with this:

postconf -e smtpd_sasl_authenticated_header=yes
postconf -e header_checks=regexp:/etc/postfix/header_checks

The following header_checks file does the magic censoring:

/^Received: from (.* \([-._[:alnum:]]+ [[.[:digit:]]{7,15}\]\)).*?(<span class="createlink"><a href="/ikiwiki.cgi?do=create&amp;from=services%2Fmail&amp;page=%3Aspace%3A" rel="nofollow">?</a>:space:</span>+).*\(Authenticated sender: ([^)]+)\).*by ([-._[:alnum:]]+) \(([^)]+)\) with (E?SMTPS?A?) id ([A-F[:digit:]]+).*/
        REPLACE Received: from [127.0.0.1] (localhost [127.0.0.1])$2(Authenticated sender: $3)${2}with $6 id $7

Note that this will expose their usernames, but that is actually useful because it allows you to track down eventual spammers.

Testing

This will try to relay an email through server example.net to the example.com domain using TLS over the submission port (587) with user name anarcat and a prompted password (-ap -pp).

swaks -f anarcat@example.net -t anarcat@example.com -s example.net -tls -p 587 -au anarcat -ap -pp

Client configuration

This should be fairly straightforward on most clients, e.g. Thunderbird should autodetect authentication on port 25, and if that is blocked, port 587 with STARTTLS should be used.

Emacs

The following customization must be performed to send email through the Emacs SMTP library:

(setq send-mail-function (quote smtpmail-send-it))
(setq smtpmail-default-smtp-server "mail.anarc.at")
(setq smtpmail-smtp-service 587)
(setq smtpmail-stream-type (quote starttls))

The first time you will send an email, it will ask you to save your credentials to ~/.authinfo. I accepted, but then went to that file and erased the password that was stored. That way Emacs remembers the password and prompts me as necessary, without storing the precious password on disk. This file is entirely customizable as well through the auth package. For example, you can store the password on disk but encrypt it using GPG, simply by encrypting the file as ~/.authinfo.gpg and removing the original. See the gpg documentation for more information.

A more detailed documentation of my client setup is in 2016-05-12-email-setup.

Postfix

To configure Postfix as a client for the above, the following configuration can be used:

postconf -e smtp_sasl_auth_enable=yes
postconf -e smtp_sasl_password_maps=hash:/etc/postfix/sasl/passwd
postconf -e smtp_sasl_security_options= 
postconf -e relayhost=[hostname]:587
postconf -e smtp_tls_security_level=encrypt
postconf -e smtpd_tls_security_level=encrypt
postmap /etc/postfix/sasl/passwd
postfix reload

The /etc/postfix/sasl/passwd file holds hostname user:pass configurations, one per line:

echo "hostname user:pass" > /etc/postfix/sasl/passwd
chown root:root /etc/postfix/sasl/passwd && chmod 600 /etc/postfix/sasl/passwd

may can be used as a security_level if we are going to send mail to other hosts which may not support security, but make sure that mails are encrypted with talking to the relayhost, maybe through the use of a smtp_tls_policy_maps.

For debugging, you can make SMTP client sessions verbose in Postfix:

smtp      unix  -       -       -       -       -       smtp -v

smtp_sasl_mechanism_filter is also very handy for debugging. For example, you can try to force the authentication mechanism to cram-md5 this way.

Note that this method stores your plaintext password on disk, which is not really desirable. It is best to configure your MUA to send email directly and ignore local emails.

Ideas

  1. ✓ test syncmaildir to replace offlineimap: would allow for an automated configuration based on SSH keys instead of passwords. DONE! this setup is documented in syncmaildir.

  2. ✓ similarly, consider using nullmailer. bremner has this setup that using SSH. nullmailer has the advantage over msmtp that it can queue emails. update: done, with a custom rsendmail, see below.

  3. ✓ alternatively, a simple sendmail wrapper that calls ssh host sendmail could do the job as well. it would need to be some restricted sendmail command, maybe like rsendmail above. but then emails can only be sent online. Update: this was implemented, with a nullmailer remote, as rsendmail.

  4. ultimately, maybe the IMAP server can send the email, through a "Outbox" folder that would slurp emails written there and send them through the SMTP server. this seems to be only supported by Courier IMAP unfortunately.

References

Delievery and retrieval over SSH

I have switched from using IMAP and SMTP to receive and deliver email to SSH as a transport mechanism. This is possible thanks to two different bits of software, syncmaildir to replace IMAP and I wrote rsendmail to replace SMTP. The former is documented in syncmaildir and the latter is documented in the upstream rsendmail documentation.

This makes it so I do not need to use clear-text passwords to deliver or retrieve email which means everything can be fully automated without writing any password on disk.

Spam filtering

Quick notes on how to configure spam filtering with Spamassassin on Debian.

apt-get install spampd
vi /etc/postfix/master.cf
postfix reload

Modifications required to master.cf:

smtp      inet  n       -       -       -       -       smtpd
        -o content_filter=smtp:127.0.0.1:10025
        -o myhostname=mx.anarcat.ath.cx
localhost:10026        inet    n       -       n       -       10      smtpd
        -o content_filter=
        -o myhostname=delivery.anarcat.ath.cx

In /etc/default/spampd:

ADDOPTS="--tagall --maxsize=1024"

... as the default max size of 64KB is just too small...

In local.cf:

use_bayes 1
bayes_path /var/cache/spampd/bayes
bayes_file_mode 0777
shortcircuit BAYES_99                spam

use_auto_whitelist 1
auto_whitelist_path /var/cache/spampd/awl
# language config, requires Mail::SpamAssassin::Plugin::TextCat
ok_languages de en es fr pt
pyzor_options --homedir /var/cache/spampd

I also load those extra modules in v310.pre:

loadplugin Mail::SpamAssassin::Plugin::Pyzor
loadplugin Mail::SpamAssassin::Plugin::AWL
loadplugin Mail::SpamAssassin::Plugin::SpamCop
loadplugin Mail::SpamAssassin::Plugin::TextCat

Pyzor needs:

apt install pyzor
sudo -u spampd pyzor --homerdir /var/cache/spampd discover

Bayes autolearning. Spampd cron jobs:

@daily sa-learn --ham --max-size=1048576 /home/anarcat/Maildir/cur/ /home/anarcat/Maildir/.ham/
@daily sa-learn --spam --max-size=0 /home/anarcat/Maildir/.junk/

Fix permissions:

cd Maildir/
chmod -R g+rX .junk/ .ham/
sudo chown -R :spampd .junk/ .ham/
# chmod +s .junk/* .ham/* # why??
chmod +s tmp
sudo chown -R :spampd cur new tmp
chmod g+rX cur new tmp -R

First training run:

[1020]anarcat@marcos:Maildir$ sudo -u spampd sa-learn --ham --progress --max-size=1048576 ~anarcat/Maildir/cur/
 55% [===============================================================                                                   ]  18.61 msgs/sec 03m03s LEFT

To manually check, use:

sudo -u spampd spamassassin -t -d -x <path>

Also, to add to whitelist:

sudo -u spampd spamassassin -t -d -x -W <path>

Also important to enable nightly rules updates:

sudo sed -i s/^CRON=./CRON=1/ /etc/default/spamassassin

This doesn't report emails to pyzor and similar services, unfortunately, see https://wiki.apache.org/spamassassin/ReportingSpam

See also: https://wiki.apache.org/spamassassin/SiteWideBayesFeedback

Todo

To improve on this, I could use the dovecot-antispam plugin, probably by piping messages into sa-learn or some wrapper script. It's unclear how permissions are managed... However, there's a spool2dir method (see upstream manpage) that could be more appropriate, as another daemon could pick up files for training, but it's not in Jessie.

Another thing I could add is the OpenPGP plugin which classifies mail according to its PGP signatures. It fetches keys on the fly and doesn't seem to check for updates. It's also old, so issues may abound.

Finally, we should keep an eye on the rspamd project which reminds me of the old dspam...

Dovecot and mail filters

Postfix is only a MTA, a Mail Transfer Agent. It takes mail from one place and puts it in another place. It doesn't allow you to read mail or do advanced filtering. For this, we need a MDA, a Mail Delivery Agent. The most robust, fastest and powerful server probably available now is Dovecot.

My Dovecot configuration is basically the default, more or less. The key part here is configuring Dovecot to use TLS, otherwise you are sending your password in cleartext all the time. For this you first need to get an X509 certificate somewhere, probably with Let's Encrypt! because it's cheap and automated. You can just reuse the certificate you create through the web server, although you will need special hooks to reload Dovecot when the cert is renewed. I use those simple symlink:

lrwxrwxrwx 1 root root 42 fév  1 19:46 /etc/dovecot/dovecot.pem -> ../letsencrypt/live/anarc.at/fullchain.pem
lrwxrwxrwx 1 root root 43 fév  1 19:42 /etc/dovecot/private/dovecot.pem -> ../../letsencrypt/live/anarc.at/privkey.pem

I also configured filtering and many more things that are documented in 2016-05-12-email-setup.

Todo

On the fly OpenPGP encryption of incoming emails?

Webmail

Yes, people like that thing. Even I like that thing now, because I want to be able to look at my mail without a full IMAP client or logging in through SSH.

I started testing Rainloop, a minimalist webmail client. It does require PHP which sucks, but is way easier to setup than Roundcube and supports mobile very well, while at the same time allowing all the great features you'd expect (sieve, contact lists, search, etc).

First part was to setup PHP. I used PHP-FPM to try to avoid the bloat associated with mod_php. I did this with:

apt install php-fpm
a2enmod proxy_fcgi setenvif
a2enconf php7.0-fpm

Then I created the following config:

<VirtualHost *:80>
    ServerName mail.anarc.at
    ServerAlias imap.anarc.at smtp.anarc.at submission.anarc.at
    Redirect / https://mail.anarc.at/
</VirtualHost>

<VirtualHost *:443>
    ServerName mail.anarc.at
    ServerAlias imap.anarc.at smtp.anarc.at submission.anarc.at
    DocumentRoot /var/www/mail.anarc.at/

    DirectoryIndex /index.php index.php 
    ProxyPassMatch ^/(.*\.php(/.*)?)$ unix:/run/php/php7.0-fpm.sock|fcgi://localhost/var/www/mail.anarc.at

    # protect rainloop configs
    <Directory /var/www/mail.anarc.at/data>
        Options -FollowSymLinks
        AllowOverride None
        <IfVersion >= 2.3>
          Require all denied
        </IfVersion>
        <IfVersion < 2.3>
          Order allow,deny
          Deny from all
        </IfVersion>
    </Directory>
<VirtualHost>

Then I setup the cert with certbot:

certbot certonly --domains mail.anarc.at,imap.anarc.at,smtp.anarc.at,submission.anarc.at --webroot --webroot-path /var/www/mail.anarc.at

... and added the following to the above vhost:

    SSLCertificateFile /etc/letsencrypt/live/mail.anarc.at/cert.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/mail.anarc.at/privkey.pem
    SSLCertificateChainFile /etc/letsencrypt/live/mail.anarc.at/chain.pem

And restarted apache of course:

service apache2 reload

Then I setup rainloop, which is disturbingly easy:

wget http://www.rainloop.net/repository/webmail/rainloop-community-latest.zip
mkdir /var/www/rainloop
unzip rainloop-latest.zip -d /var/www/rainloop

Then I visited the admin page (/?admin) and made the following changes:

That must be one of the simplest webapp install I've seen, considering the complexity of this thing. Bravo!

Created . Edited .