Basic mail server configuration
This page documents the configuration of the mail server. There is no user-facing documentation yet.
Running a mail server can be surprisingly hard, considering how long the protocols have been around. However, I believe it is something that can be managed by any moderately experience administrator, and in fact could be better automated if we could figure out a simple set of standard configurations. Hopefully, this page should cover the latest good configuration for small servers.
DNS
One of the trickiest parts of configuring email is to setup DNS
correctly. You must have an A
record that points to your server's IP
address that is always up to date, and a reverse PTR
record that
points to the same name. Otherwise a lot of servers will bounce your
outgoing emails.
Furthermore, some providers block certain IP blocks as being "consumer" addresses that are never supposed to send email. There is little you can do against this but hope for the best or find a provider with "clean addresses".
It is also possible that the address you are given by your provider is blocked because it sent spam in the past: in this case your only hope is to get the address unblocked or have your provider give you another IP address.
SPF
SPF records are fairly simple: they specify policies on the servers allowed to send email for a specific domain. They are an extension of the traditional "reverse DNS must match" kind of policies.
In my case, it was simply a matter of saying that my main server is responsible for all outgoing mail:
@ TXT "v=spf1 a mx -all"
Breaking it down, this means:
v=spf1
: this is SPF version 1 (RFC7208)a
:A
records for the domain name (e.g. theA
record ofanarc.at
) are allowedMX
:MX
records for the domain name (e.g. theMX
record ofanarc.at
, currently marcos) are allowed
Because the mail exchanger (which receives email) is the same as the
outgoing email server, this is sufficient. If those services would be
split up (for example with a separate machine sending email like a
mailing list server), this would need to be expanded, for example by
including a:lists.example.com
.
Configurations can be tested with those tools:
- Vamsoft policy tester or the
- DMARC analyzer
- mxtoolbox.com
- intodns.com
- dt (golang)
DKIM
DKIM is a standard (RFC6376) to sign email headers and prevent forgeries.
First install the DKIM server and tools:
apt install opendkim opendkim-tools
Create some configuration directories:
mkdir -p /etc/opendkim/keys
In /etc/opendkim.conf
, we add those lines:
# write socket inside postfix's chroot
Socket local:/var/spool/postfix/opendkim/opendkim.sock
# key/host mappings
SigningTable refile:/etc/opendkim/signing.table
KeyTable /etc/opendkim/key.table
# Hosts to generate signatures for
InternalHosts /etc/opendkim/internal.hosts
# Hosts to ignore when verifying signatures, same as above
ExternalIgnoreList /etc/opendkim/internal.hosts
We would have used Domain
if we had only one domain, but since we
have many, we need those tables. First one, is signing.table
:
*@anarc.at marcos
*@orangeseeds.org marcos
First part is a pattern, second is a key that is then found in
key.table
:
marcos anarc.at:marcos:/etc/opendkim/keys/marcos.private
First field is the signing.table
key. Second is field is a
colon-separated list of fields: the domain name, a selector (can be
anything, but we picked the hostname), and the private key file.
Then generate the private key and DNS record
opendkim-genkey --directory=/etc/opendkim/keys/ --selector=marcos --domain=anarc.at --verbose
The private key is the .private
file specified above, and the DNS
record is written in a .txt
file. The latter should be included in
the zone file:
$INCLUDE "/etc/opendkim/keys/marcos.txt"
Unfortunately, that fails with an obscure permission denied
, maybe
because it's outside of the normal bind directories and/or
apparmor. Instead, copy-paste the content, which will look something
like this:
marcos._domainkey IN TXT ( "v=DKIM1; h=sha256; k=rsa; "
"p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAubF5LI+R0lroVTItfbbs714+BW3waB34fUZ7wJT6Vrj3QVNg82bjIAL9u+WMOGt4okNi/QhjtoofqeUTVJycULiu1YXn6yQ8Dqrvm4s4uO8mwErOPV2mIplyRVwLEmS5zCw4UcVTdyDnPHqbzrRXAJfwJ3qFwwLwRraLDuuzqv+RX+mb5/s4VgjAXXAFHzDOdhcj7kGk8CzxXW"
"ZWt8W7ilJUSoRoslBoA/jj5TMUU/xtbtSV0kZUVf8Y+0IKuxJHTlSrDqJ/PcMJZqMHtRY2ZQPtzaGPeFLJRN1J4U+krqerJiCc+n7P0KBS/yPb0H24mbWGP2WFxou3s3XdYeGSiQIDAQAB" ) ; ----- DKIM key marcos for anarc.at
Then this will tell other servers we always sign our mails, and refuse unsigned emails:
; enforce DKIM on all mails from this domain, deprecated
_adsp._domainkey IN TXT "dkim=all"
Note that the above "ADSP" policy is now discouraged in favor of DMARC.
Fix permissions on the key files:
chmod o-rw /etc/opendkim/keys
chown opendkim:root /etc/opendkim/keys/*
The DKIM server is then restarted and checked:
service opendkim restart
opendkim-testkey -d anarc.at -s marcos -vv
The keys not secure
message means you are not using DNSSEC.
Then create the directory with proper permissions:
mkdir /var/spool/postfix/opendkim
chown opendkim /var/spool/postfix/opendkim
Allow postfix to access the opendkim socket:
adduser postfix opendkim
And restart dkim again:
service opendkim restart
Then hook it up in postfix:
postconf -e milter_protocol=6
postconf -e milter_default_action=accept
postconf -e smtpd_milters=local:opendkim/opendkim.sock
postconf -e non_smtpd_milters=local:opendkim/opendkim.sock
If emails are double-signed, add receive_override_options=no_milters
to loopback smtpd
servers in master.cf
, for example:
localhost:10026 inet n - n - 10 smtpd
-o smtpd_tls_security_level=none
-o content_filter=
-o myhostname=delivery.anarc.at
-o receive_override_options=no_milters
Then postfix must be reloaded:
postfix reload
Then follow the test procedure, below.
Adding a new domain
Debian.org added support for DKIM in 2020. To configure this on my side, I had to do the following, on top of the above...
add this line to
signing.table
:*@debian.org marcos-debian.anarcat.user
add this line to
key.table
:marcos-debian.anarcat.user debian.org:marcos-debian.anarcat.user:/etc/opendkim/keys/marcos-debian.anarcat.user.private
Yes, that's quite a mouthful! That magic selector is long in that way because it needs a special syntax (specifically the
.anarcat.user
suffix) for Debian to be happy. The-debian
string is to tell me where the key is published. Themarcos
prefix is to remind me where the private is used.generate the key with:
opendkim-genkey --directory=/etc/opendkim/keys/ --selector=marcos-debian.anarcat.user --domain=debian.org --verbose
This creates the DNS record in
/etc/opendkim/keys/marcos-debian.anarcat.user.txt
(alongside the private key in.key
).restart
opendkim
:service opendkim restart
The DNS record will look something like this:
marcos-debian.anarcat.user._domainkey IN TXT ( "v=DKIM1; h=sha256; k=rsa; " "p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAtKzBK2f8vg5yV307WAOatOhypQt3ANQ95iDaewkVehmx42lZ6b4PzA1k5DkIarxjkk+7m6oSpx5H3egrUSLMirUiMGsIb5XVGBPFmKZhDVmC7F5G1SV7SRqqKZYrXTufRRSne1eEtA31xpMP0B32f6v6lkoIZwS07yQ7DDbwA9MHfyb6MkgAvDwNJ45H4cOcdlCt0AnTSVndcl" "pci5/2o/oKD05J9hxFTtlEblrhDXWRQR7pmthN8qg4WaNI4WszbB3Or4eBCxhUdvAt2NF9c9eYLQGf0jfRsbOcjSfeus0e2fpsKW7JMvFzX8+O5pWfSpRpdPatOt80yy0eqpm1uQIDAQAB" ) ; ----- DKIM key marcos-debian.anarcat.user for debian.org
The
"p=MIIB..."
string needs to be joined together, without the quotes and thep=
, and sent in a signed email tochanges@db.debian.org
:-----BEGIN PGP SIGNED MESSAGE----- dkimPubKey: marcos.anarcat.user MIIB[...] -----BEGIN PGP SIGNATURE----- [...]
Wait a few minutes for DNS to propagate. You can check if they have with:
host -t TXT marcos-debian.anarcat.user._domainkey.debian.org nsp.dnsnode.net
(
nsp.dnsnode.net
being one of theNS
records of thedebian.org
zone.)
If all goes well, the tests (below) should pass when sending from your
server as anarcat@debian.org
.
Testing
Test messages can be sent to dkimvalidator, mail-tester.com,
or check-auth@verifier.port25.com
. Those tools will run Spamassassin
on the received emails and report the results. What you are looking
for is:
-0.1 DKIM_VALID
: Message has at least one valid DKIM or DK signature-0.1 DKIM_VALID_AU
: Message has a valid DKIM or DK signature from author's domain-0.1 DKIM_VALID_EF
: Message has a valid DKIM or DK signature from envelope-from domain
If one of those is missing, then you are doing something wrong and
your "spamminess" score will be worse. The latter is especially tricky
as it validates the "Envelope From", which is the MAIL FROM:
header
as sent by the originating MTA, which you see as from=<>
in the
postfix lost.
The following will happen anyways, as soon as you have a signature, that's normal:
0.1 DKIM_SIGNED
: Message has a DKIM or DK signature, not necessarily valid
And this might happen if you have a ADSP record but do not correctly sign the message with a domain field that matches the record:
1.1 DKIM_ADSP_ALL
No valid author signature, domain signs all mail
That's bad and will affect your spam core badly. I fixed that issue by using a wildcard key in the key table:
--- a/opendkim/key.table
+++ b/opendkim/key.table
@@ -1 +1 @@
-marcos anarc.at:marcos:/etc/opendkim/keys/marcos.private
+marcos %:marcos:/etc/opendkim/keys/marcos.private
References used:
- Ubuntu documentation
- Debian wiki
- linode tutorial, also recommends rotating keys every 6 months
- jak-linux: uses rspamd instead of opendkim, and PostSRSd
- Gio's tutorial
Another test tool is https://mxtoolbox.com/emailhealth.
DMARC
I am using this simple, non-restrictive DMARC policy:
_dmarc IN TXT "v=DMARC1;p=none;pct=100;rua=mailto:postmaster@anarc.at"
Breaking it down, this means:
v=DMARC
: version numberp=none
: policy, do nothing (could bequarantine
orreject
)pct=100
: to which ratio of emails does the policy apply to (all emails)rua=...
: where to send failure reports
I am not clear yet on how that interacts with DKIM and SPF, but that seems like a safe way to start.
The above will lead to reports landing in your mailbox. To parse them, you can use the dmarc-cat tool which I packaged for Debian.
Postfix
I am using the Postfix server. I find it easier to configure than the other common mail server, Exim, which I never bothered learning.
Debian does a good job at configuring Postfix for you, when you install the package. It does ask a bit too many questions for my taste, but it does help with the grunt of the work. Here are, some pointers to the answers you want to give:
- General configuration:
Internet site
- Mail name: use default
- Root recipient: your email
- Synchronous updates:
no
(requested this question to be removed from the Postfix package, see bug #832953)
There are literally hundreds of configuration settings available in Postfix, the best is to the excellent Postfix documentation for further guides. Some of them are simplified below.
Postfix opportunistic TLS
This configures Postfix to offer a X509 certificate on inbound and
outbound SMTP connections but also accept unencrypted connections as a
fallback, in main.cf
:
# configure certificates
smtpd_tls_cert_file = /etc/letsencrypt/live/anarc.at/fullchain.pem
smtpd_tls_key_file = /etc/letsencrypt/live/anarc.at/privkey.pem
smtp_tls_cert_file = ${smtpd_tls_cert_file}
smtp_tls_key_file = ${smtpd_tls_key_file}
# opportunistic encryption
smtpd_tls_security_level = may
smtp_tls_security_level = may
See the TLS_README
and smtp_tls_security_level
for more
information.
This patch is required to disable TLS when a content_filter
is
configured (see FILTER_README
and below).
diff --git a/postfix/master.cf b/postfix/master.cf
index b0ed875..6329c70 100644
--- a/postfix/master.cf
+++ b/postfix/master.cf
@@ -12,7 +12,9 @@
smtp inet n - - - - smtpd
-o content_filter=smtp:127.0.0.1:10025
-o myhostname=mx.anarc.at
+ -o smtp_tls_security_level=none
localhost:10026 inet n - n - 10 smtpd
+ -o smtpd_tls_security_level=none
-o content_filter=
-o myhostname=delivery.anarc.at
submission inet n - - - - smtpd
Without this patch, local delivery would hang during my tests.
MTA-STS
The above has now somewhat been standardized as RFC 8461, "SMTP MTA Strict Transport Security".
For me this involved creating the following file in my well-known
directory (/var/www/.well-known/mta-sts.txt
):
version: STSv1
mode: testing
mx: marcos.anarc.at
max_age: 86400
Then a set of new record needs to be added to DNS:
mta-sts IN CNAME marcos
_smtp._tls IN TXT "v=TLSRPTv1;rua=mailto:postmaster@anarc.at"
_mta-sts 300 IN TXT "v=STSv1; id=20190225113927Z;"
The id
field should be a unique string that changes when there's a
policy change. I picked the format output by date +%Y%m%d%H%M%SZ
.
Then a vhost needs to be created for the above file to be accessible:
<VirtualHost *:80>
ServerName mta-sts.anarc.at
ServerAlias mta-sts.orangeseeds.org
DocumentRoot /var/www/html/
#Redirect / https://mta-sts.anarc.at/
</VirtualHost>
<VirtualHost *:443>
ServerName mta-sts.anarc.at
ServerAlias mta-sts.orangeseeds.org
DocumentRoot /var/www/html/
#Use common-letsencrypt-ssl mta-sts.anarc.at
</VirtualHost>
This needs to be authenticated with certbot
of course and uncomment
the redirect and Use
lines when done:
rndc reload
service apache2 reload
certbot certonly --webroot --webroot-path /var/www/html/ -d mta-sts.anarc.at -d mta-sts.orangeseeds.org
Then the configuration can be checked on aykevl.nl or hardenize.com.
This only works for incoming email. For outgoing email, Postfix needs to be able to check the TLS policy (which it could do with smtp_tls_policy_maps) but the MTA-STS checks are not supported in the daemon itself. Thankfully, there's a third-party daemon called postfix-mta-sts-resolver which can do this. Unfortunately it wasn't in Debian when I worked on this (since then solved, see bug 917366) so I didn't get to deploy that just yet.
References:
Postfix SASL configuration
This allows users to submit mails using their regular IMAP credentials.
Dovecot
We don't want to configure a new authentication system for Postfix, so we delegate to the already existing one, which is Dovecot.
In the default configuration, the following patch is required to Dovecot to enable a control socket exposed to Postfix:
diff --git a/dovecot/conf.d/10-auth.conf b/dovecot/conf.d/10-auth.conf
index 1c59eb4..187b262 100644
--- a/dovecot/conf.d/10-auth.conf
+++ b/dovecot/conf.d/10-auth.conf
@@ -97,7 +97,7 @@
# plain login digest-md5 cram-md5 ntlm rpa apop anonymous gssapi otp skey
# gss-spnego
# NOTE: See also disable_plaintext_auth setting.
-auth_mechanisms = plain
+auth_mechanisms = plain login
##
## Password and user databases
diff --git a/dovecot/conf.d/10-master.conf b/dovecot/conf.d/10-master.conf
index e3d6260..5068100 100644
--- a/dovecot/conf.d/10-master.conf
+++ b/dovecot/conf.d/10-master.conf
@@ -93,9 +93,11 @@ service auth {
}
# Postfix smtp-auth
- #unix_listener /var/spool/postfix/private/auth {
- # mode = 0666
- #}
+ unix_listener /var/spool/postfix/private/auth {
+ mode = 0660
+ user = postfix
+ group = postfix
+ }
# Auth process is run as this user.
#user = $default_internal_user
Then notify Dovecot of the change:
service dovecot reload
Postfix server
This enables SASL authentication with Dovecot. It ensures passwords are only sent over a secure channel and censors the original IP address.
--- a/postfix/main.cf
+++ b/postfix/main.cf
@@ -46,8 +46,9 @@ home_mailbox = Maildir/
#mailbox_command = /usr/bin/procmail -a "$EXTENSION"
smtpd_recipient_restrictions = reject_unlisted_recipient,
permit_mynetworks,
+ permit_sasl_authenticated,
reject_non_fqdn_recipient,
reject_unauth_destination,
check_policy_service inet:127.0.0.1:10023,
reject_rbl_client zen.spamhaus.org
In other words, make sure that permit_sasl_authenticated
is
added to smtpd_recipient_restrictions
.
Then you need to hook Dovecot SASL authentication into Postfix and make sure it is not offered in cleartext:
postconf -e smtpd_sasl_type=dovecot
postconf -e smtpd_sasl_path=private/auth
postconf -e smtpd_tls_auth_only=yes
Then enable the submission port (587) in master.cf
:
diff --git a/postfix/master.cf b/postfix/master.cf
index f4f096d..b0ed875 100644
--- a/postfix/master.cf
+++ b/postfix/master.cf
@@ -15,11 +15,11 @@ smtp inet n - - - - smtpd
localhost:10026 inet n - n - 10 smtpd
-o content_filter=
-o myhostname=delivery.anarc.at
-#submission inet n - - - - smtpd
-# -o smtpd_tls_security_level=encrypt
-# -o smtpd_sasl_auth_enable=yes
-# -o smtpd_client_restrictions=permit_sasl_authenticated,reject
-# -o milter_macro_daemon_name=ORIGINATING
+submission inet n - - - - smtpd
+ -o smtpd_tls_security_level=encrypt
+ -o smtpd_sasl_auth_enable=yes
+ -o smtpd_client_restrictions=permit_sasl_authenticated,reject
+ -o milter_macro_daemon_name=ORIGINATING
smtps inet n - - - - smtpd
-o smtpd_tls_wrappermode=yes
# -o smtpd_sasl_auth_enable=yes
As an extra, you can protect your users' privacy by hiding their IP address with this:
postconf -e smtpd_sasl_authenticated_header=yes
postconf -e header_checks=regexp:/etc/postfix/header_checks
The following header_checks
file does the magic censoring:
/^Received: from (.* \([-._[:alnum:]]+ [[.[:digit:]]{7,15}\]\)).*?(<span class="createlink">:space:</span>+).*\(Authenticated sender: ([^)]+)\).*by ([-._[:alnum:]]+) \(([^)]+)\) with (E?SMTPS?A?) id ([A-F[:digit:]]+).*/
REPLACE Received: from [127.0.0.1] (localhost [127.0.0.1])$2(Authenticated sender: $3)${2}with $6 id $7
Note that this will expose their usernames, but that is actually useful because it allows you to track down eventual spammers.
Testing
This will try to relay an email through server example.net
to the
example.com
domain using TLS over the submission port (587
) with
user name anarcat
and a prompted password (-ap -pp
).
swaks -f anarcat@example.net -t anarcat@example.com -s example.net -tls -p 587 -au anarcat -ap -pp
Client configuration
This should be fairly straightforward on most clients, e.g. Thunderbird should autodetect authentication on port 25, and if that is blocked, port 587 with STARTTLS should be used.
Emacs
The following customization must be performed to send email through the Emacs SMTP library:
(setq send-mail-function (quote smtpmail-send-it))
(setq smtpmail-default-smtp-server "mail.anarc.at")
(setq smtpmail-smtp-service 587)
(setq smtpmail-stream-type (quote starttls))
The first time you will send an email, it will ask you to save your
credentials to ~/.authinfo
. I accepted, but then went to that file
and erased the password that was stored. That way Emacs remembers the
password and prompts me as necessary, without storing the precious
password on disk. This file is entirely customizable as well through
the auth package. For example, you can store the password on disk
but encrypt it using GPG, simply by encrypting the file as
~/.authinfo.gpg
and removing the original. See the
gpg documentation for more information.
A more detailed documentation of my client setup is in 2016-05-12-email-setup.
Postfix
To configure Postfix as a client for the above, the following configuration can be used:
postconf -e smtp_sasl_auth_enable=yes
postconf -e smtp_sasl_password_maps=hash:/etc/postfix/sasl/passwd
postconf -e smtp_sasl_security_options=
postconf -e relayhost=[hostname]:587
postconf -e smtp_tls_security_level=encrypt
postconf -e smtpd_tls_security_level=encrypt
postmap /etc/postfix/sasl/passwd
postfix reload
The /etc/postfix/sasl/passwd
file holds hostname user:pass
configurations, one per line:
echo "hostname user:pass" > /etc/postfix/sasl/passwd
chown root:root /etc/postfix/sasl/passwd && chmod 600 /etc/postfix/sasl/passwd
may
can be used as a security_level
if we are going to send mail
to other hosts which may not support security, but make sure that
mails are encrypted with talking to the relayhost
, maybe through the
use of a smtp_tls_policy_maps
.
For debugging, you can make SMTP client sessions verbose in Postfix:
smtp unix - - - - - smtp -v
smtp_sasl_mechanism_filter
is also very handy for debugging. For
example, you can try to force the authentication mechanism to
cram-md5
this way.
Note that this method stores your plaintext password on disk, which is not really desirable. It is best to configure your MUA to send email directly and ignore local emails.
Ideas
✓ test syncmaildir to replace offlineimap: would allow for an automated configuration based on SSH keys instead of passwords. DONE! this setup is documented in syncmaildir.
✓ similarly, consider using nullmailer. bremner has this setup that using SSH. nullmailer has the advantage over msmtp that it can queue emails. update: done, with a custom rsendmail, see below.
✓ alternatively, a simple
sendmail
wrapper that callsssh host sendmail
could do the job as well. it would need to be some restricted sendmail command, maybe likersendmail
above. but then emails can only be sent online. Update: this was implemented, with a nullmailer remote, as rsendmail.ultimately, maybe the IMAP server can send the email, through a "Outbox" folder that would slurp emails written there and send them through the SMTP server. this seems to be only supported by Courier IMAP unfortunately.
References
Delivery and retrieval over SSH
I have switched from using IMAP and SMTP to receive and deliver email to SSH as a transport mechanism.
This was originally implemented with syncmaildir (SMD) to replace IMAP, but I have now switched to mbsync. I also wrote rsendmail to replace SMTP.
The my old SMD setup is documented in syncmaildir and the latter is documented in the upstream rsendmail documentation.
This makes it so I do not need to use clear-text passwords to deliver or retrieve email which means everything can be fully automated without writing any password on disk.
Update: I am abandoning this approach, as it requires exposing SSH to the universe, something I want to avoid now. Looking into client certs instead.
Spam filtering
Quick notes on how to configure spam filtering with Spamassassin on Debian.
apt-get install spampd
vi /etc/postfix/master.cf
postfix reload
Modifications required to master.cf:
smtp inet n - - - - smtpd
-o content_filter=smtp:127.0.0.1:10025
-o myhostname=mx.anarcat.ath.cx
localhost:10026 inet n - n - 10 smtpd
-o content_filter=
-o myhostname=delivery.anarcat.ath.cx
In /etc/default/spampd
:
ADDOPTS="--tagall --maxsize=1024"
... as the default max size of 64KB is just too small...
In local.cf:
use_bayes 1
bayes_path /var/cache/spampd/bayes
bayes_file_mode 0777
shortcircuit BAYES_99 spam
use_auto_whitelist 1
auto_whitelist_path /var/cache/spampd/awl
# language config, requires Mail::SpamAssassin::Plugin::TextCat
ok_languages de en es fr pt
pyzor_options --homedir /var/cache/spampd
I also load those extra modules in v310.pre:
loadplugin Mail::SpamAssassin::Plugin::Pyzor
loadplugin Mail::SpamAssassin::Plugin::AWL
loadplugin Mail::SpamAssassin::Plugin::SpamCop
loadplugin Mail::SpamAssassin::Plugin::TextCat
Pyzor needs:
apt install pyzor
sudo -u spampd pyzor --homerdir /var/cache/spampd discover
Bayes autolearning. Spampd cron jobs:
@daily sa-learn --ham --max-size=1048576 /home/anarcat/Maildir/cur/ /home/anarcat/Maildir/.ham/
@daily sa-learn --spam --max-size=0 /home/anarcat/Maildir/.junk/
Fix permissions:
cd Maildir/ &&
chmod -R g+rX . .junk/ .ham/ &&
sudo chown -R :spampd .junk/ .ham/ &&
chmod g+s .junk/* .ham/* &&
chmod g+s tmp &&
sudo chown -R :spampd cur new tmp &&
chmod g+rX cur new tmp -R
First training run:
anarcat@marcos:Maildir$ sudo -u spampd sa-learn --ham --progress --max-size=1048576 ~anarcat/Maildir/cur/
55% [=============================================================== ] 18.61 msgs/sec 03m03s LEFT
Junk training run:
sudo -u spampd sa-learn --spam --progress --max-size=0 ~anarcat/Maildir/.junk/cur/
To manually check, use:
sudo -u spampd spamassassin -t -d -x <path>
Also, to add to whitelist:
sudo -u spampd spamassassin -t -d -x -W <path>
Also important to enable nightly rules updates:
sudo sed -i s/^CRON=./CRON=1/ /etc/default/spamassassin
This doesn't report emails to pyzor and similar services, unfortunately, see https://wiki.apache.org/spamassassin/ReportingSpam
See also: https://wiki.apache.org/spamassassin/SiteWideBayesFeedback
DMARC / SPF pre-checks
apt install opendmarc
Choose "no" when prompted to configure the database.
The default config is used except for those additions:
--- a/opendmarc.conf
+++ b/opendmarc.conf
@@ -64,7 +64,8 @@ PublicSuffixList /usr/share/publicsuffix/public_suffix_list.dat
## either in the configuration file or on the command line. If an IP
## address is used, it must be enclosed in square brackets.
#
-Socket local:/run/opendmarc/opendmarc.sock
+#Socket local:/run/opendmarc/opendmarc.sock
+Socket local:/var/spool/postfix/opendmarc/opendmarc.sock
## Syslog { true | false }
## default "false"
@@ -102,7 +103,7 @@ Syslog true
## specific file mode on creation regardless of the process umask. See
## umask(2) for more information.
#
-UMask 0002
+UMask 0007
## UserID user[:group]
## default (none)
@@ -112,3 +113,14 @@ UMask 0002
## the named userid unless an alternate group is specified.
#
UserID opendmarc
+
+# ignore submissions through the submission port
+IgnoreAuthenticatedClients true
+# reject stuff like missing From:
+RequiredHeaders true
+# add headers to processed emails
+SoftwareHeader true
+# ignore existing SPF headers that might be spoofed
+SPFIgnoreResults false
+# do the SPF checks as well
+SPFSelfValidate true
Then that socket needs to be added to the postfix configuration:
--- a/postfix/main.cf
+++ b/postfix/main.cf
@@ -119,8 +119,8 @@ compatibility_level = 2
# OpenDKIM
milter_protocol = 6
milter_default_action = accept
-smtpd_milters = local:opendkim/opendkim.sock
-non_smtpd_milters = local:opendkim/opendkim.sock
+smtpd_milters = local:opendkim/opendkim.sock local:opendmarc/opendmarc.sock
+non_smtpd_milters = local:opendkim/opendkim.sock local:opendmarc/opendmarc.sock
Finally, add the postfix user to the opendmarc group so it can access the socket, and restart both servers:
adduser postfix opendmarc
systemctl restart postfix opendmarc
Note that this setup is "soft", in that it will add headers after checking the email but will not outright reject the emails. My idea is to run this in advisory mode for a while and eventually enable rejections, when I'm satisfied this is safe. The magic setting is:
RejectFailures true
Some ideas here are from https://jan.wildeboer.net/2022/09/Email-3-TheRest/.
Todo
To improve on this, I could use the dovecot-antispam plugin, probably by piping messages into sa-learn or some wrapper script. It's unclear how permissions are managed... However, there's a spool2dir method (see upstream manpage) that could be more appropriate, as another daemon could pick up files for training, but it's not in Jessie.
Another thing I could add is the OpenPGP plugin which classifies mail according to its PGP signatures. It fetches keys on the fly and doesn't seem to check for updates. It's also old, so issues may abound.
Finally, we should keep an eye on the rspamd project which reminds me of the old dspam...
Dovecot and mail filters
Postfix is only a MTA
, a Mail Transfer Agent. It takes mail from one
place and puts it in another place. It doesn't allow you to read mail
or do advanced filtering. For this, we need a MDA, a Mail Delivery
Agent. The most robust, fastest and powerful server probably available
now is Dovecot.
My Dovecot configuration is basically the default, more or less. The key part here is configuring Dovecot to use TLS, otherwise you are sending your password in cleartext all the time. For this you first need to get an X509 certificate somewhere, probably with Let's Encrypt! because it's cheap and automated. You can just reuse the certificate you create through the web server, although you will need special hooks to reload Dovecot when the cert is renewed. I use those simple symlink:
lrwxrwxrwx 1 root root 42 fév 1 19:46 /etc/dovecot/dovecot.pem -> ../letsencrypt/live/anarc.at/fullchain.pem
lrwxrwxrwx 1 root root 43 fév 1 19:42 /etc/dovecot/private/dovecot.pem -> ../../letsencrypt/live/anarc.at/privkey.pem
I also configured filtering and many more things that are documented in 2016-05-12-email-setup.
Client certs
Creating a self-signed ed25519 private CA
We copied over the /usr/lib/ssl/openssl.cnf
config file. We have
actually tried a configuration-file-less setup, but it breaks down
when you start using the openssl ca
command, necessary to revoke
certificates.
Create basic directories and files:
mkdir private certs req newcerts
echo 00 > serial
We don't have a serial crlnumber
but if we would, we would start
with:
printf 00 > crlnumber
Generate the CA secret key:
openssl genpkey -algorithm ed25519 -out private/cakey.pem -aes256
ED25519 instructions were taken from this post.
Then generate a self-signed cert:
openssl req -subj "/CN=ca.anarc.at/" -key private/cakey.pem -out cacert.pem -new -x509 -days 3650 -reqexts v3_ca -config openssl.cnf
Alternatives include OpenVPN's easy-rsa and cfssl, which also has a puppet module.
Client key and certificate creation
Then the client key is generated, on the client, again with (but without encryption):
openssl genpkey -algorithm ed25519 -out angela.anarc.at.key
The openssl.cnf
file for the certificate request:
[client-cert]
keyUsage = cRLSign, keyCertSign
extendedKeyUsage = clientAuth
[req]
distinguished_name = dn
prompt = no
x509_extensions = client-cert
[dn]
CN = angela.anarc.at
emailAddress = anarcat
Create the CSR with:
openssl req -key angela.anarc.at.key -out angela.anarc.at.csr -config openssl.cnf -new
Copy the CSR and CRT files to the CA server and sign the request with:
openssl ca -days 365 -in req/test.anarc.at.csr -out certs/test.anarc.at.crt
... from this guide. Alternatively, this can be done without the
CA, with the lower-level x509
command:
openssl x509 -req -in req/angela.anarc.at.csr -CA cacert.pem -CAkey private/cakey.pem -days 365 -out certs/angela.anarc.at.crt
Again, from RHEL.
The cert can be checked with:
openssl x509 -text < certs/angela.anarc.at.crt
... and:
openssl verify -CAfile cacrt.pem certs/angela.anarc.at.crt
Generate the CRL file, currently just the cert because we haven't revoked anything yet:
cp cacert.pem cacrl.pem
Postfix server configuration
Before:
submission inet n - y - - smtpd
-o header_checks=regexp:/etc/postfix/header_authenticated_redaction
-o smtpd_tls_security_level=encrypt
-o smtpd_sasl_auth_enable=yes
-o smtpd_client_restrictions=permit_sasl_authenticated,reject
-o milter_macro_daemon_name=ORIGINATING
After:
submission inet n - y - - smtpd
-o header_checks=regexp:/etc/postfix/header_authenticated_redaction
-o milter_macro_daemon_name=ORIGINATING
-o smtpd_tls_security_level=encrypt
-o smtpd_tls_fingerprint_digest=sha256
-o smtpd_tls_ask_ccert=yes
-o smtpd_recipient_restrictions=permit_tls_clientcerts,reject
-o smtpd_relay_restrictions=permit_tls_clientcerts,reject
-o relay_clientcerts=hash:/etc/postfix/client-certs-fingerprints
We were hoping to use permit_tls_all_clientcerts like this:
-o tls_append_default_CA=no
-o smtpd_tls_CAfile=/etc/ssl/ca/cacrl.pem
-o smtpd_recipient_restrictions=permit_tls_all_clientcerts,reject
-o smtpd_relay_restrictions=permit_tls_all_clientcerts,reject
but that silly thing doesn't support certificate revocation: it looks
like the CRL part of the cacrl.pem
file is ignore. So it's
impossible to remove client certificates, so we need to use the static
list. An alternative is to use Dovecot 2.3 submission functionality,
since the CRL works there.
The certificates list is created with:
rm /etc/postfix/client-certs-fingerprints
for cert in certs/* ; do
printf "%s %s\n" "$(
openssl x509 -in $cert -noout -pubkey |
openssl pkey -pubin -outform DER |
openssl dgst -sha256 -c |
sed 's/.*= //'
)" $cert >> /etc/postfix/client-certs-fingerprints
done
And of course the map needs to be rehashed each time:
postmap /etc/postfix/client-certs-fingerprints
Note that this does include revoked certificates as well, so you kind of have to manually skip the bad certs. (TODO.)
Then this should work:
swaks --tls --tls-cert ~/.config/x509/angela.anarc.at2.crt --tls-key ~/.config/x509/angela.anarc.at.key -s marcos.anarc.at -t anarcat@torproject.org -p 587
Postfix client configuration
This is relatively simple. First, we create a new transport to encapsulate our configuration because we have other relays with either no client TLS authentication or different authentication mechanisms (e.g. passwords):
default_transport = smtptlsc:
relayhost = smtp.anarc.at:587
Then we configure that transport as such:
smtptlsc unix - - y - - smtp
-o smtp_tls_CAfile=/etc/ssl/certs/ca-certificates.crt
-o smtp_tls_cert_file=/etc/ssl/private/angela.anarc.at.crt
-o smtp_tls_key_file=/etc/ssl/private/angela.anarc.at.key
-o smtp_tls_fingerprint_digest=sha256
-o smtp_tls_security_level=secure
The cert need to be copied in /etc/ssl/private
and the key given to
the ssl-cert
group.
Note that this is done in the profile::postfix::satellite
class
(satellite.pp) and the above configuration might be out of
date. Also note that we use the whole smtp_tls_CAfile
instead of the
CApath
because the latter doesn't work in the chroot.
To test this, try to relay mail locally:
mail anarcat@example.com -s test < /dev/null
Turn up the logging level on the client:
smtp_tls_loglevel=2
... and the server:
smtpd_tls_loglevel=2
... if you have issues.
Dovecot configuration
The dovecot SSL configuration docs are quite limited. So we're using another guide instead. The also have limited SSL docs...
Enable TLS verification in conf.d/10-ssl.conf
:
ssl_ca = </etc/ssl/ca/cacrl.pem
ssl_verify_client_cert = yes
ssl_cert_username_field = email
ssl_require_crl = yes
Create conf.d/auth-tls.conf.ext
:
# Take the username from client's SSL certificate, using
# X509_NAME_get_text_by_NID() which returns the subject's DN's
# CommonName.
auth_ssl_username_from_cert = yes
# Space separated list of wanted authentication mechanisms:
# plain login digest-md5 cram-md5 ntlm rpa apop anonymous gssapi otp
# gss-spnego
# NOTE: See also disable_plaintext_auth setting.
auth_mechanisms = plain login external
passdb {
driver = passwd-file
args = scheme=PLAIN username_format=%u /etc/dovecot/users-external
mechanisms = external
override_fields = nopassword
}
userdb {
# <doc/wiki/AuthDatabase.Passwd.txt>
driver = passwd
# [blocking=no]
#args =
# Override fields from passwd
#override_fields = home=/home/virtual/%u
}
Note that the above uses the normal user database so the user need to exist on the system as well.
Then include that in conf.d/10-auth.conf
, and comment out the other includes:
#!include auth-system.conf.ext
!include auth-tls.conf.ext
If we'd like to keep Postfix using passwords, we could do:
protocol !smtp {
auth_ssl_require_client_cert=yes
}
... but since we're going to use TLS there too, that makes obviously no sense. Plus it gets rid of the weird SASL shim between the two, woot.
During this deployment, SSH-based IMAP connexions still work, which is pretty fantastic.
This can be tested with:
openssl s_client -CAfile /etc/ssl/certs/ca-certificates.crt -verify 4 -cert angela.anarc.at.crt -key angela.anarc.at.key -starttls imap -connect localhost:imap
That will not actually do any IMAP query (although you could try
10 AUTHENTICATE EXTERNAL
to confirm login works). Better try the
swiss-army-knife of everything:
curl --cert angela.anarc.at.crt --key angela.anarc.at.key --login-options AUTH=EXTERNAL imaps://imap.anarc.at
This should list your folders. Use -v
for more debugging if things fail.
At first, this was failing with:
dovecot: imap-login: Disconnected: Aborted login by logging out (no auth attempts in 0 secs): user=<>, [...]
... and that was because I was missing the AUTH=EXTERNAL
option.
To debug issues with TLS, turn on Dovecot's verbose logging in
conf.d/10-logging.conf
:
verbose_ssl = yes
You can now test revocation with:
openssl ca -config openssl.cnf -revoke certs/angela.anarc.at.crt -gencrl > crl.pem
cat cacert.pem crl.pem > cacrl.pem
service dovecot restart
Note that, by default, that damn crl expires after 30 days, you'll
probably want to bump that expiry date (with default_crl_days=3650
in openssl.conf
or with the -crldays
option) and then:
openssl ca -config openssl.cnf -gencrl > crl.pem
cat cacert.pem crl.pem > cacrl.pem
service dovecot restart
And now the above curl
command should fail. Notice how dovecot needs
a kick after revocation, a reload
might be sufficient as well.
Another guide has instructions on how to disable TLS certs for some services, e.g. if Postfix would still require SASL auth.
Adding a new satellite
To add a new satellite to this setup, you need to generate a new key
on the client, and a CSR, based on the following config. Typically,
you only need to do this for Postfix, so this can more easily be done
in /etc/postfix/x509
(and that is where Puppet configures Postfix to
look for certs).
This is how tubman was configured. First, make the directory:
mkdir /etc/postfix/x509
cd /etc/postfix/x509
Then create openssl.conf
:
[client-cert]
keyUsage = cRLSign, keyCertSign
extendedKeyUsage = clientAuth
[req]
distinguished_name = dn
prompt = no
x509_extensions = client-cert
[dn]
CN = tubman.anarc.at
emailAddress = tubman-mail
Then generate the private key and the CSR:
openssl genpkey -algorithm ed25519 -out client.key
openssl req -key client.key -out client.csr -config openssl.cnf -new
Then copy that over to the CA in /etc/ssl/ca/req/tubman.anarc.at.csr
and sign the request:
openssl ca -days 365 -in req/tubman.anarc.at.csr -out certs/tubman.anarc.at.crt
Then regenerate the list of trusted certs:
rm /etc/postfix/client-certs-fingerprints
for cert in certs/* ; do
printf "%s %s\n" "$(
openssl x509 -in $cert -noout -pubkey |
openssl pkey -pubin -outform DER |
openssl dgst -sha256 -c |
sed 's/.*= //'
)" $cert >> /etc/postfix/client-certs-fingerprints
done
postmap /etc/postfix/client-certs-fingerprints
Add the profile::postfix::satellite
class to the node and it should
be able to send mail. Test with:
mail -s test anarcat@example.com < /dev/null
Easy-RSA CA notes
I tested building a CA with easy-rsa but ended up not using it because my end goal is to do this in Puppet, so I couldn't rely on such a large third-party tool directly. Plus, I didn't think it supported ed25519 keys at first (it does though!).
To get started with easy-rsa:
apt install easy-rsa
make-cadir easyrsa
./easyrsa init-pki
To make a ED25519 CA, add those to vars
:
set_var EASYRSA_ALGO ed
set_var EASYRSA_CURVE ed25519
Then:
./easyrsa build-ca
That prompts for a password then runs something like:
["openssl", "req", "-config", "/etc/ssl/easyrsa/pki/5892315c/temp.4ff4e933", "-utf8", "-new", "-key", "/etc/ssl/easyrsa/pki/5892315c/temp.fa90dc32", "-keyout", "/etc/ssl/easyrsa/pki/5892315c/temp.fa90dc32", "-out", "/etc/ssl/easyrsa/pki/5892315c/temp.591dfebb", "-x509", "-days", "3650", "-sha256", "-passin", "file:/etc/ssl/easyrsa/pki/5892315c/temp.c5904947"],
Interesting facts:
- it generates a key on the fly
-utf-8
-x509
-days 3650
A client cert can be created with:
./easyrsa build-client-full angela.anarc.at
To get the emailAddress
field, the vars
need to be modified to
have:
set_var EASYRSA_DN "org"
set_var EASYRSA_REQ_EMAIL "anarcat"
Then the guide mentions "exporting the combined CA+CRL" with:
./easyrsa gen-crl
That runs:
["openssl", "ca", "-config", "/etc/ssl/easyrsa/pki/d1f80f21/temp.7ec4b8d2", "-utf8", "-gencrl", "-out", "/etc/ssl/easyrsa/pki/d1f80f21/temp.12bd618a"],
... and generates /etc/ssl/easyrsa/pki/crl.pem
but this is odd
because the guide also says it generates a pki/ca+crl.pem
file,
which cannot be found. That can be fixed with:
cat /etc/ssl/easyrsa/pki/{ca.crt,crl.pem} > /etc/ssl/easyrsa/pki/ca+crl.pem
Also, interestingly, it uses that ca+crl.pem
file in the Postfix
configuration, with permit_tls_all_clientcerts
, which leads me to
think it might be possible to avoid listing all fingerprints. To be
tested/confirmed.
Conversion effect on performance
I have sampled the last ~100 mbsync
runs, which is from April 08
01:52:56 to April 09 14:42:37 (non-inclusively), with:
journalctl -u mbsync.service --user -n 1000 | grep Consumed | sed '/avr 09 14:42:37/,$d;s/.*Consumed //;s/s CPU time.//' > timings-over-ssh
This gave me 132 samples:
$ wc -l timings-over-ssh
132 timings-over-ssh
The average CPU usage was:
$ awk 'BEGIN { sum = 0; count = 0 } { sum += $1; count++ } END { print sum / count}' < timings-over-ssh
2.70843
Things seem faster. The evidence is a bit anecdotal now, as I have only 4 samples, but there is already a clear reduction in CPU usage:
$ journalctl -u mbsync.service --user -n 1000 | grep Consumed | sed -n '/avr 09 14:42:37/,$p' | sed 's/.*Consumed //;s/s CPU time.//' | awk 'BEGIN { sum = 0; count = 0 } { sum += $1; count++ } END { print sum / count}'
2.1356
This could be because the TLS key exchange is better optimized than SSH. And indeed, a casual look at the logs seem to suggest it was taking 4 seconds to sync before and after, so it could just be an accounting issue.
Given that this work was done for security reasons and not optimization reasons, I'm satisfied with the results since there's no meaningful regression.
Remaining work
- TODO: renewals, switch to easyrsa fully?
- TODO: generate and distribute certs with Puppet
Todo
On the fly OpenPGP encryption of incoming emails?
Webmail
Yes, people like that thing. Even I like that thing now, because I want to be able to look at my mail without a full IMAP client or logging in through SSH.
I started testing Rainloop, a minimalist webmail client. It does require PHP which sucks, but is way easier to setup than Roundcube and supports mobile very well, while at the same time allowing all the great features you'd expect (sieve, contact lists, search, etc).
First part was to setup PHP. I used PHP-FPM to try to avoid the bloat
associated with mod_php
. I did this with:
apt install php-fpm
a2enmod proxy_fcgi setenvif
a2enconf php7.0-fpm
Then I created the following config:
<VirtualHost *:80>
ServerName mail.anarc.at
ServerAlias imap.anarc.at smtp.anarc.at submission.anarc.at
Redirect / https://mail.anarc.at/
</VirtualHost>
<VirtualHost *:443>
ServerName mail.anarc.at
ServerAlias imap.anarc.at smtp.anarc.at submission.anarc.at
DocumentRoot /var/www/mail.anarc.at/
DirectoryIndex /index.php index.php
ProxyPassMatch ^/(.*\.php(/.*)?)$ unix:/run/php/php7.0-fpm.sock|fcgi://localhost/var/www/mail.anarc.at
# protect rainloop configs
<Directory /var/www/mail.anarc.at/data>
Options -FollowSymLinks
AllowOverride None
<IfVersion >= 2.3>
Require all denied
</IfVersion>
<IfVersion < 2.3>
Order allow,deny
Deny from all
</IfVersion>
</Directory>
<VirtualHost>
Then I setup the cert with certbot:
certbot certonly --domains mail.anarc.at,imap.anarc.at,smtp.anarc.at,submission.anarc.at --webroot --webroot-path /var/www/mail.anarc.at
... and added the following to the above vhost:
SSLCertificateFile /etc/letsencrypt/live/mail.anarc.at/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/mail.anarc.at/privkey.pem
SSLCertificateChainFile /etc/letsencrypt/live/mail.anarc.at/chain.pem
And restarted apache of course:
service apache2 reload
Then I setup rainloop, which is disturbingly easy:
wget http://www.rainloop.net/repository/webmail/rainloop-community-latest.zip
mkdir /var/www/rainloop
unzip rainloop-latest.zip -d /var/www/rainloop
Then I visited the admin page (/?admin
) and made the following
changes:
- General:
- Disallow additional accounts
- Domains:
- Disabled gmail
- Added wildcard domain (
*
) with:- Use short login (both IMAP and SMTP, to avoid having to enter the domain
- SSL/TLS security (on IMAP, not SMTP, as I want to deliver without credentials locally)
- SMTP: localhost
- Security:
- local proxying for external images
- Allow 2-step verification (untested)
- Changed admin password
- Require verification of SSL certificate used
- Disallow self signed certificates
- Login:
- Try to determine user domain (unchecked)
- Contacts:
- Enable contacts
- Type: SQLite
That must be one of the simplest webapp install I've seen, considering the complexity of this thing. Bravo!
Mailing lists
NOTE: this is mostly moot. I have uninstalled Mailman 3 locally, but ended up reusing those docs to update the torproject.org mailman 2 server instead, see the TPA lists documentation instead.
I naively thought I could go old school and replace Facebook with email (even though I actually never used Facebook). I figured, heck, mailing lists, I know that, I'll just install Mailman 3 in Debian and be done with it.
How wrong can one be. First bug I found was that stretch doesn't have mailman 3, it's only in backports. But then the dependencies in the package are all out of whack (bug #919145, bug #920304). The workaround is simple:
apt install python3-alembic python3-sqlalchemy python3-pymysql python3-mysqldb
apt install -t stretch-backports mailman3-full
Then I found out that the mailman3-web interface is simply
uninstallable when using MySQL (reported as bug #921128). So I
just used the sqlite3
backend, which is promising to cause
delightful problems when interoperating with the mailman3
package,
running MySQL.
dpkg-reconfigure mailman3-web
This, incidentally, allows us to have the web server (Apache2) automatically configured, but we won't do that - we'll configure it by hand.
Then Postfix needs to be configured:
owner_request_special = no
transport_maps = hash:/etc/postfix/transport
hash:mailman3/postfix_lmtp
local_recipient_maps = proxy:unix:passwd.byname $alias_maps hash:mailman3/postfix_lmtp
relay_domains = ${{$compatibility_level} < {2} ? {$mydestination} : {}} hash:mailman3/postfix_domains
This differs from the configuration suggested in the README because the postfix daemons are usually chrooted (reported as bug #921445). This is then symlinked in place:
touch /var/spool/postfix/mailman3/postfix_domains /var/spool/postfix/mailman3/postfix_lmtp
chown list:list /var/spool/postfix/mailman3/postfix_*
postmap /var/spool/postfix/mailman3/postfix_domains /var/spool/postfix/mailman3/postfix_lmtp
ln -s /var/spool/postfix/mailman3/postfix_domains /var/spool/postfix/mailman3/postfix_lmtp /var/lib/mailman3/data/
And, of course, postfix reloaded:
postfix reload
The data_dir
needs to be changed in the mailman config
(/etc/mailman3/mailman.cfg
), and while you're there change the
site_owner
as well:
site_owner: anarcat+register@anarc.at
data_dir: /var/spool/postfix/mailman3/
Then we create an Apache config (because the default one kind of sucks):
<VirtualHost *:80>
ServerName lists.anarc.at
#Redirect / https://lists.anarc.at/
DocumentRoot /var/www/html/
</VirtualHost>
<VirtualHost *:443>
ServerName lists.anarc.at
#Use common-letsencrypt-ssl lists.anarc.at
DocumentRoot /var/www/html/
RedirectMatch ^/$ /mailman3/
Include /etc/mailman3/apache.conf
</VirtualHost>
Reload apache:
service apache2 reload
A certificate is obtained, after creating the domain of course:
certbot certonly -w /var/www/html -d lists.anarc.at --webroot
Once the cert is enabled, uncomment the Redirect
and Use
lines and
relaod apache again:
service apache2 reload
Finally, a Posterius super user needs to be created:
django-admin createsuperuser --pythonpath /usr/share/mailman3-web --settings settings --username anarcat --email anarcat+register@anarc.at
That will prompt for a password, head over to
https://lists.anarc.at/mailman3/ to login. This will ask for an email
confirmation, which should confirm your email system somewhat
works. Follow that. Then you must configure the Domains
to make sure
they match the hostname. After you can create a test mailing list and
try delivery.
Resolved Issues
I have found out that I had masquerade_domains
enabled in my
main.cf
: bad idea. It made a fool out of myself in bug #921137,
where I complained that Mailman would rewrite emails in or out and
that would break unsubscribe links and other stuff. I simply disabled
that line for now, we'll see what breaks.
Other issues:
bug #919145 - mailman3: stretch-backports dependencies can not be satisfied with python3-alembic from backports. workaround: force-install stable versions (see above)
bug #920304 - mailman3-web: mailman3web / django does not like python3-pymysqlbug #921128: mailman3-web fails to initialize mysql: Specified key was too long. workaround: sqlite3 or upgrade to buster.
mailing lists archives would simply not work. this was fixed by adding this to
mailman.py
:# The following lines are specific to mailing lists archiving using # HyperKitty. They require 'python3-mailman-hyperkitty' to be installed # and will produce errors otherwise. # # If you don't want to use HyperKitty, please comment them out. [archiver.hyperkitty] class: mailman_hyperkitty.Archiver enable: yes configuration: /etc/mailman3/mailman-hyperkitty.cfg
The password in
mailman-hyperkitty.cfg
must match theMAILMAN_ARCHIVER_KEY
inmailman-web.py
. I would have expected the Debian packages to handle that, but I might have screwed it up."subscription message" template ineffective: sends empty message (not reported, but bug #919970 is related) - can't reproduce??
code linting issues in Mailman core made my other MR fail (MR #444)
no "invite" subscription mechanism (mailman bug #510) - workaround: custom API script to invite users seem to work. usage:
export MAILMAN_PASSWORD=$(sed -n '/^admin_pass:/{s/.*: //;p}' mailman.cfg) mm3-invite-user.py test@lists.anarc.at anarcat@example.com
Update: fixed in MR 678 (3.3.2 and later)
confirmation mail subject is not translatable, subjects are not templatable (mailman bug #541). workaround: patch to make Mailman not send the email so we can do it ourselves (mailman MR 452))
Update: fixed in MR 678 (3.3.2 and later)
translated french templates are missing (mailman bug #540) - workaround: add french templates as needed through the web GUI (adding in filesystem fails because of mailman bug #535 - but I backported MR 442 so lookups now work. templates can be added in any of:
$template_dir/lists/test.example.com/it/foo.txt
$template_dir/domains/example.com/it/foo.txt
$template_dir/site/it/foo.txt
<source_dir>/templates/it/foo.txt
... where
$template_dir
is/var/lib/mailman3
and<source_dir>
is/usr/lib/python3/dist-packages/mailman/
. This means templates can be created on the filesystem for individual lists without going through the GUI. updates are shipped upstream in MR #446.Update: fixed.
unicode templates gets mangled (mailman bug #542) - workaround: switch French to utf-8 (MR #443), change default encoding to utf-8 (MR #445) and change
default_language
tofr
inmailman.cfg
. then switch the list to french:# su -s /bin/sh -c "mailman shell -l test.lists.anarc.at" list Welcome to the GNU Mailman shell The variable 'm' is the test.lists.anar.at mailing list >>> m.preferred_language = 'fr' >>> commit()
Update: probably fixed?
Remaining issues
- UI french translation is missing - workaround: partial translation
started, with
.pot
file but still missing compiling and shipping the.mo
file (MR 453)
Tested
- delivery
- (mass) subscription
- reply
- unsubscribe
- archives
- mail from web
- invites (fail + workaround)
- translation (fail + workaround)
Future work
- mailman3 pgp plugin - interesting if not promising project, but requires many patches to anything from all the Mailman components to PGPy itself.
Disaster recovery
If the mail server goes down, I currently have one or two fall back options:
- migrate email to my shared hosting Koumbit.org account, they support DKIM, SPF, and some DMARC, generally delivers well to Google, some issues with Microsoft (hotmail/office), no spam filtering
- use a Riseup email
Other providers that could replace this service include:
- Cloudflare: requires CF hosting your DNS
- Fastmail: 5$/mth for 30GB and your own domain
- immerda.ch also does host emails with your own domain
new VM setup
This, ideally, should be in Puppet, but in the disaster recover, this shiny puppet server was actually down.
Bootstrap
apt install postfix postfix-pcre
apt install dovecot-imapd dovecot-sieve dovecot-managesieved
apt install postgrey
apt install opendkim opendkim-tools
apt install opendmarc
apt install chrony certbot
Append this to /etc/postfix/main.cf
:
mailbox_command = /usr/lib/dovecot/dovecot-lda -a "$RECIPIENT"
mailbox_size_limit = 0
message_size_limit = 50240000
milter_default_action = accept
milter_protocol = 6
non_smtpd_milters = local:opendkim/opendkim.sock local:opendmarc/opendmarc.sock
smtp_address_preference = ipv4
smtp_dns_support_level = dnssec
smtp_tls_cert_file=${smtpd_tls_cert_file}
smtp_tls_key_file=${smtpd_tls_key_file}
smtp_tls_mandatory_ciphers = high
smtp_tls_security_level = dane
smtpd_helo_required = yes
smtpd_helo_restrictions = reject_non_fqdn_helo_hostname, reject_invalid_helo_hostname
smtpd_milters = local:opendkim/opendkim.sock local:opendmarc/opendmarc.sock
smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unknown_client_hostname, reject_non_fqdn_recipient, reject_unauth_destination, reject_unlisted_recipient, check_policy_service inet:127.0.0.1:10023, reject_rbl_client zen.spamhaus.org
smtpd_tls_cert_file = /etc/letsencrypt/live/colette.anarc.at/fullchain.pem
smtpd_tls_key_file = /etc/letsencrypt/live/colette.anarc.at/privkey.pem
opendkim:
# write socket inside postfix's chroot
Socket local:/var/spool/postfix/opendkim/opendkim.sock
# key/host mappings
SigningTable refile:/etc/opendkim/signing.table
KeyTable /etc/opendkim/key.table
# Hosts to generate signatures for
InternalHosts /etc/opendkim/internal.hosts
# Hosts to ignore when verifying signatures, same as above
ExternalIgnoreList /etc/opendkim/internal.hosts
Populate:
mkdir -p /var/spool/postfix/opendmarc /etc/opendkim/keys /var/spool/postfix/opendkim
chown opendmarc /var/spool/postfix/opendmarc
chown opendkim /var/spool/postfix/opendkim
adduser postfix opendkim
adduser postfix opendmarc
Config dovecot in conf.d/99-$hostname.conf
:
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
mail_location = maildir:~/Maildir
restore /etc/postfix/master.cf
, /etc/postfix/mydestination
and
/etc/aliases
from backups. consider restoring main.cf
as
well. hell, just restore all of /etc/postfix from backups at this
point right? make sure you get /etc/postfix/header_authenticated_redaction
Get a cert:
certbot certonly --standalone -d $(hostname -f)
Enable the cert:
ln -sf /etc/letsencrypt/live/colette.anarc.at/privkey.pem /etc/dovecot/private/dovecot.key
ln -sf /etc/letsencrypt/live/colette.anarc.at/cert.pem /etc/dovecot/private/dovecot.pem
Add the register user:
adduser --disabled-password register
spamd install and config
adduser anarcat spampd
sed -i s/^CRON=./CRON=1/ /etc/default/spamassassin
echo 'ADDOPTS="--maxsize=1024"' >> /etc/default/spampd
cat >> /etc/spamassassin/local.cf
cd Maildir/ &&
chmod -R g+rX . .junk/ .ham/ &&
sudo chown -R :spampd .junk/ .ham/ &&
chmod +gs .junk/* .ham/* &&
chmod +gs tmp &&
sudo chown -R :spampd cur new tmp &&
chmod g+rX cur new tmp -R