Installing a mailserver on Debian 8 – Part 8: Quota, rules, aliases and other settings

How to install a complete mailserver on Debian 8, featuring Postfix, Dovecot, MySQL, Spamassassin, ClamAV, Roundcube and Fail2ban.

~ This is the howto that actually works. ~

Part 1: Introduction
Part 2: Preparations: Apache, Let’s Encrypt, MySQL and phpMyAdmin
Part 3: MTA: Postfix
Part 4: IMAP server: Dovecot
Part 5: Web interface: Roundcube
Part 6: Spam filtering: SpamAsasssin
Part 7: Antivirus: ClamAV and ClamSMTP
Part 8: Quota and other Roundcube settings
Part 9: Using mail with a remote IMAP client (i.e. Thunderbird)
Part 10: Counter brute-force attacks with Fail2ban
Part 11: Sources, config files, colouring and comments

On this page

Quota
Rules
Aliases
Other Roundcube settings

Comments are on the last page.

Sidestepping the roadmap for a bit
Sidestepping the roadmap for a bit

This page contains some odds and ends.

Quota

Mail storage quota are defined in /etc/dovecot/conf.d/90-quota.conf:

plugin {
  quota_rule = *:storage=5M
  quota_rule2 = Trash:storage=+1M
  quota_grace = 10%%
  quota_status_overquota = "552 5.2.2 Mailbox is full"
}

Line 5 (quota_status_overquota) is optional. There are a lot more optional settings.
For this example I set a 5 megabyte storage limit plus 1 megabyte for the Trash folder, with a grace value of 10%. (The grace value is the amount of slack you allow the user.) 10G would mean 10 gigabytes.

In the backends section:

plugin {
  quota = maildir:User quota
}

Read the comments for some explanation on quotas. More info: http://wiki2.dovecot.org/Quota

In /etc/dovecot/conf.d/10-mail.conf:

mail_plugins = $mail_plugins quota

In /etc/dovecot/conf.d/20-imap.conf:

mail_plugins = $mail_plugins imap_quota

Reload Dovecot:

# dovecot reload

Log out of Roundcube and log back in. Roundcube will now show an icon in the bottom left corner to indicate how full the mailbox is.

Mailbox quotum indicator
Mailbox quotum indicator

User rules

In the part about spam filtering we set up the Sieve filtering system. It is possible for the users to create their own Sieve rules from within Roundcube, for example to automatically move mails with certain content or a specified sender to a folder.

First we must enable the Sieve plugin. Open /etc/roundcube/config.inc.php and add the Managesieve plugin:

// List of active plugins (in plugins/ directory)
$config['plugins'] = array(
'archive',
'zipdownload',
'password',
'managesieve',
);

The default /etc/roundcube/plugins/managesieve/config.inc.php in the Debian backports repo at the moment of writing is quite empty. In /usr/share/roundcube/plugins/managesieve there’s a symlink to that file but there’s really no point (since it’s empty). So what we’ll do is unlink the file and copy the distribution file to the Roundcube install:

# unlink /usr/share/roundcube/plugins/managesieve/config.inc.php
# cp /usr/share/roundcube/plugins/managesieve/config.inc.php.dist /etc/roundcube/plugins/managesieve/config.inc.php

In the Roundcube web interface click Settings; you should now find a button labeled Filters.

Roundcube has a concept of ‘Filter sets’. You can create sets of filters but these are not groupings: only one set can be active at a time. So I tend to ignore the Filter sets and I suggest you tell your users about this as it can be quite confusing.

Now with filters!
Now with filters!

To create a filter click the + button in the Filters column. Select your parameters. If you want to overwrite the global spam rule we created earlier add an extra action that says ‘Stop evaluating rules’.

Creating a user filter
Creating a user filter

Aliases

Aliases work much the same as regular addresses in terms of setup: they are mentioned in Postfix’s main.cf and described in a config file telling Postfix how to connect to the database and what to look for.

We already set up the database in Part 2 so let’s connect it.

In /etc/postfix/main.cf:

virtual_alias_maps = mysql:/etc/postfix/mysql_virtual_aliases.cf

In /etc/postfix/mysql_virtual_aliases.cf:

user = mailman
password = P@ssw0rd
hosts = 127.0.0.1
dbname = postfix
query = SELECT target FROM aliases WHERE source = '%s'

Enter an alias in the database:

# mysql -u root -p
mysql> INSERT INTO `postfix`.`aliases` (`id`, `source`, `target`) VALUES (NULL, 'helpdesk@example.com', 'tinus@example.com');
mysql> quit

Or from phpMyAdmin:

mailsvr044b

Test the connection with

# postmap -q helpdesk@example.com mysql:/etc/postfix/mysql_virtual_aliases.cf

This should result in one answer:

tinus@example.com

Non-existent aliases should result in no answer. If you get an error check the permission the user you specified in mysql_virtual_aliases.cf has got on the aliases table in the Postfix database. Check for typos.

I deliberately named the aliases table’s fields ‘source’ and ‘target’ so it’s easy to distinguish between them. These names are by no means mandatory and you are free to design as elaborate a database and a query as you like.

Other Roundcube Settings

To provide users with default settings you would edit the file /etc/roundcube/defaults.inc.php. In it you can set defaults for things such as which spell checker to use, which language to default to, and so on.

Installing a mailserver on Debian 8 – Part 6: Spamfiltering: SpamAssassin

How to install a complete mailserver on Debian 8, featuring Postfix, Dovecot, MySQL, Spamassassin, ClamAV, Roundcube and Fail2ban.

~ the howto that actually works ~

Part 1: Introduction
Part 2: Preparations: Apache, Let’s Encrypt, MySQL and phpMyAdmin
Part 3: MTA: Postfix
Part 4: IMAP server: Dovecot
Part 5: Web interface: Roundcube
Part 6: Spam filtering: SpamAsasssin
Part 7: Antivirus: ClamAV and ClamSMTP
Part 8: Quota and other Roundcube settings
Part 9: Using mail with a remote IMAP client (i.e. Thunderbird)
Part 10: Counter brute-force attacks with Fail2ban
Part 11: Sources, config files, colouring and comments

On this page

Installing Spamassassin
Testing the spamfilter
Moving spam to the Junk folder automatically
Automatically subscribe users to the special IMAP folders
Sieve
Create the actual filter
Automatically delete spam after 30 days

Comments are on the last page.

On this page
On this page

For this project I used Spamassassin. Spamassassin is less granular and comprehensive than ASSP but in this case I only had a handful of users and Spamassassin’s not bad.

Note that spamfiltering is done in two phases:
1. Spam is identified upon entering the system.
2. Spam is delivered in a dedicated folder in the user’s account.

These phases are separate things. Spamassassin identifies and tags spam while Dovecot’s Sieve plugin allows automatically putting the spam in the user’s Junk folder. We’ll do spam identification first.

Installing Spamassassin

# aptitude install spamassassin spamc

Create a user for spamc. From the spamc man page: “Spamc is the client half of the spamc/spamd pair. It should be used in place of “spamassassin” in scripts to process mail. It will read the mail from STDIN, and spool it to its connection to spamd, then read the result back and print it to STDOUT. Spamc has extremely low overhead in loading, so it should be much faster to load than the whole spamassassin program.”

# adduser spamd --disabled-login

Just press enter at each question.

Have Spamassassin start on boot:

# systemctl enable spamassassin.service

Have Spamassassin update its rules automatically every night. In /etc/default/spamassassin:

CRON=1

Configuration of Spamassassin is done in /etc/spamassassin/local.cf. The defaults are ok. After the filter has had a chance to learn what is spam and what is not (say, after two weeks of use) add

use_bayes_rules 1

to /etc/spamassassin/local.cf. This will have Spamassassin actually use Bayesian the rules it learned.

Info: http://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Conf.html

Start the spam filter:

# service spamassassin start

Tell Postfix we’re running a content filter. In /etc/postfix/master.cf find the line that reads

smtp      inet  n       -       -       -       -       smtpd

and directly underneath it add:

  -o content_filter=spamassassin

Note that the line beginning with -o must start with one more whitespaces. My Syntax Highlighter seems to be stripping them off. If it doesn’t reloading Postfix will say: “/usr/sbin/postconf: fatal: invalid type field”. (Again, countless hours of fun.)

At the end of /etc/postfix/master.cf add:

spamassassin unix -     n       n       -       -       pipe
  user=spamd argv=/usr/bin/spamc -f -e /usr/sbin/sendmail -oi -f ${sender} ${recipient}

Again note that the second line must start with one or more whitespaces.

# service spamassassin start
# postfix reload

Once you’ve installed and configured Postfix send a test mail to and from your account in Roundcube and make sure it still works.
If it doesn’t remove the newest entries from /etc/postfix/master.cf, restart Postfix and try again. See if you have made a typo and make sure restarting Postfix does not throw any errors. Only if sending and receiving works continue.

Testing the spamfilter

Send a mail to your recipient with the GTUBE. The GTUBE is EICAR for spamfilters: it is a special string of characters which spamfilters will recognize as spam for the purpose of testing. More on the GTUBE: http://spamassassin.apache.org/gtube/

So send a mail with this string in its body:

XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X

Open the mail from Roundcube. It should start with:
Spam detection software, running on the system “mail.localdomain”,
has identified this incoming email as possible spam.

Incoming spam
Incoming spam

The same mail not containing the GTUBE should not contain that warning.

I like to see the original spam message, not a message containing the spam mail as attachment so I set Spamassassin to tag the mail header and not put it inside an other mail as attachment. In /etc/spamassassin/local.cf change:

report_safe 0
# service spamassassin reload

Send another GTUBE mail, open it in Roundcube and view its source.

View the e-mail's source
View the e-mail’s source

It should read:

X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mail.localdomain
X-Spam-Flag: YES
X-Spam-Level: **************************************************
X-Spam-Status: Yes, score=1000.0 required=5.0 tests=GTUBE,HTML_MESSAGE
        autolearn=no autolearn_force=no version=3.4.0
X-Spam-Report: 
        * 1000 GTUBE BODY: Generic Test for Unsolicited Bulk Email
        *  0.0 HTML_MESSAGE BODY: HTML included in message

Moving spam to the Junk folder automatically

This is actually part of something larger, namely a mail filtering system. Using filters (also known as rules) you can apply actions to mail using logic, for example move mail that’s tagged as spam to the Junk folder.

Automatically subscribe users to the special IMAP folders

First we need to make sure the IMAP client is subscribed to the folder the spam will be moved to.

If you open Roundcube you’ll notice that there are two folders: Inbox and Sent. I like to automatically subscribe users to a bunch of special IMAP folders so they’ll always be there. Open up /etc/dovecot/conf.d/15-mailboxes.conf and have it read:

namespace inbox {
  mailbox Drafts {
    special_use = \Drafts
    auto = subscribe
  }
  mailbox Junk {
    special_use = \Junk
    auto = subscribe
  }
  mailbox Trash {
    special_use = \Trash
    auto = subscribe
  }
  mailbox Archive {
    special_use = \Archive
    auto = subscribe
  }
  mailbox Sent {
    special_use = \Sent
    auto = subscribe
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
}

Apart from copying and pasting the above text also read the commentary in that file. It’s quite useful.

Do

# dovecot reload

and log out and back in to Roundcube because the special folders are created at login.

Automatically subscribed to special folders
Automatically subscribed to special folders

Sieve

Dovecot has a plugin called Sieve that handles rules. Sieve actually is a programming language designed for mail filtering rules.

Let’s enable it.
In /etc/dovecot/conf.d/15-lda.conf:

protocol lda {
  mail_plugins = $mail_plugins sieve
}

in /etc/dovecot/conf.d/20-lmtp.conf:

protocol lmtp {
  mail_plugins = $mail_plugins sieve
  postmaster_address = tinus@example.com
}

It’s a good idea to have a local postmaster address to prevent mail loops. Also it is required by the smtp rfc (though we’re technically not using smtp but lmtp here – still it’s a good idea).

In /etc/dovecot/conf.d/90-sieve.conf:

plugin {
  sieve = ~/.dovecot.sieve
  sieve_dir = ~/sieve
}
# service dovecot restart

Test if Sieve is running:

# telnet localhost 4190

If you get

Connected to localhost.
Escape character is '^]'.
"IMPLEMENTATION" "Dovecot Pigeonhole"
"SIEVE" "fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave"
"NOTIFY" "mailto"
"SASL" "PLAIN LOGIN"
"VERSION" "1.0"
OK "Dovecot ready."

then Sieve is working. Tap Enter three times to exit.

If you get

Trying 127.0.0.1...
telnet: Unable to connect to remote host: Connection refused

then Sieve is not working. Check your settings, restart the Dovecot service and try again.

Do

doveconf -n

to check for typos and such.

doveconf -n can help spot the errors.
doveconf -n can help spot the errors.

It is possible to create default sieve rules. This has one huge drawback: as soon as the user creates her own sieve rules (for example she sets an out of office autoreply) the default rules are no longer carried out even after deleting the personal rule.

Set up the spam rule globally

We’ll set up a global spam rule that will move spam to the user’s Junk folder automatically.

In /etc/dovecot/conf.d/90-sieve.conf set:

sieve_after = /etc/dovecot/sieve/spamfilter.sieve

The above folder and filename are arbitrary but they seem logical to me.

Note: the sieve_global_dir directive is not related to this feature.

Why sieve_after? If you’d use sieve_before then any applicable (to that particular mail) rules the user would have would not be executed. By using sieve_after the user’s rules are executed first so they can override the sieve_after rules.

Create the folder:

mkdir /etc/dovecot/sieve

In /etc/dovecot/sieve/spamfilter.sieve:

require ["fileinto"];
# rule:[SPAM]
if header :contains "X-Spam-Level" "*" {
        fileinto "Junk";
}

This rule checks for a header called ‘X-Spam-Level’. If its value contains * then the message is stored in the folder called Junk. Note that ‘**’ and ‘***’ also contain ‘*’ so you can set the amount of spam-certainty required for a message to be regarded spam. The scale is 1 to 50, 50 being the most certain.

Reload Dovecot and send a test message containing the GTUBE string to check if it works.

Spam is delivered in the Junk folder
Spam is delivered in the Junk folder

Automatically delete spam after 30 days

Using doveadm we can automatically delete all mail in the Junk folder older than a certain time, for example 30 days.

Doveadm is Dovecot’s administration utility. Read man doveadm for more information on doveadm. Other relevant reading for this case: man doveadm-search and man doveadm-search-query.

Let’s say you want to search all documents in all inboxes older than four hours:

# doveadm search -A mailbox Inbox savedbefore 4h
tinus@example.com 74b4db2757f9f05661350000595b2a1f 1
tinus@example.com 74b4db2757f9f05661350000595b2a1f 2

-A means to look in all mailboxes, not just the one we specify. Remember the iteration query? This is what that is good for. Had you been using a static user query in /etc/dovecot/dovecot-sql.conf.ext then doveadm search -A would have thrown an error.

We don’t want to search the Inbox however and we want mail older than 30 days:

# doveadm search -A mailbox Junk savedbefore 30d
tinus@example.com 4adeeb2ca5c1ca565c3b000038e0142a 1
tinus@example.com 4adeeb2ca5c1ca565c3b000038e0142a 2
tinus@example.com 4adeeb2ca5c1ca565c3b000038e0142a 3
$ man doveadm-search-query

tells you more about the query format.

We don’t just want to search but actually delete the mails. (Do use search first to verify you get the results you were expecting.)

# doveadm expunge -A mailbox Junk savedbefore 30d

You don’t want to run this manually every day. Let’s schedule it:

# crontab -e

Add this line:

@hourly /usr/bin/doveadm expunge -A mailbox Junk savedbefore 30d

If you don’t want the cronjob’s output mailed to you change to

@hourly /usr/bin/doveadm expunge -A mailbox Junk savedbefore 30d > /dev/null 2>&1

This will run the command every hour. Change to

@daily /usr/bin/doveadm expunge -A mailbox Junk savedbefore 30d > /dev/null 2>&1

To run it once every day.

Installing a mailserver on Debian 8 – Part 11: Sources, config files, colouring and comments

How to install a complete mailserver on Debian 8, featuring Postfix, Dovecot, MySQL, Spamassassin, ClamAV, Roundcube and Fail2ban.

~ the howto that actually works ~

Part 1: Introduction
Part 2: Preparations: Apache, Let’s Encrypt, MySQL and phpMyAdmin
Part 3: MTA: Postfix
Part 4: IMAP server: Dovecot
Part 5: Web interface: Roundcube
Part 6: Spam filtering: SpamAsasssin
Part 7: Antivirus: ClamAV and ClamSMTP
Part 8: Quota and other Roundcube settings
Part 9: Using mail with a remote IMAP client (i.e. Thunderbird)
Part 10: Counter brute-force attacks with Fail2ban
Part 11: Sources, config files, colouring and comments

On this page

Sources
Config files
Colouring Picture
Comments

All done!
All done!

I hope you’ve enjoyed my howto on installing a mailserver on Debian. If this article has been of use to you feel free to click the ads to generate a few cents of revenue or even consider a small donation using the PayPal button in the upper right corner of this page.

Please post comments or questions the comment section on this page. Note that I can’t answer every individual question and more technical questions should generally be asked in the mailing lists of the software in question.

~~~

If you’ve made it this far I salute you. To reward you for your perseverance you may print out the colouring picture below for an hour of colouring fun \o/ If you scan it and mail to dekapitein [at] this site’s domain I will post it here 🙂

Sources

Official documentation:
Debian
Postfix
Dovecot
Fail2ban
ClamAV
ClamSMTP
SpamAssassin
Thunderbird
Let’s Encrypt

Inspiring sources:
PostfixBasicSetupHowto
PostfixVirtualMailBoxClamSmtpHowto
EasyEngine: Sieve Mail Filtering Setup
Debian Mail Server with Postfix and Dovecot
Server world: Install Dovecot
BinaryTides: Setup a mail server with Postfix and Dovecot on Ubuntu / Debian
DigitalOcean: How To Configure a Mail Server Using Postfix, Dovecot, MySQL, and SpamAssassin
RoseHosting: How to set-up server-side email filtering with Dovecot Sieve and Roundcube on a CentOS 6 VPS

Config files

This file contains all settings of the files we’ve worked with in this howto. You may use it for reference in case you get stuck. It is not intended for production.

Config files

Colouring Picture

Picture courtesy of krewinkelkrijst.nl. Click for a larger version!
colouring-vorkbaard-nl-mailserver

Installing a mailserver on Debian 8 – Part 10: Security: counter brute-force attacks with Fail2ban

How to install a complete mailserver on Debian 8, featuring Postfix, Dovecot, MySQL, Spamassassin, ClamAV, Roundcube and Fail2ban.

~ the howto that actually works ~

Part 1: Introduction
Part 2: Preparations: Apache, Let’s Encrypt, MySQL and phpMyAdmin
Part 3: MTA: Postfix
Part 4: IMAP server: Dovecot
Part 5: Web interface: Roundcube
Part 6: Spam filtering: SpamAsasssin
Part 7: Antivirus: ClamAV and ClamSMTP
Part 8: Quota and other Roundcube settings
Part 9: Using mail with a remote IMAP client (i.e. Thunderbird)
Part 10: Counter brute-force attacks with Fail2ban
Part 11: Sources, config files, colouring and comments

On this page

Installation
Block random address spammers
Users entering a wrong username/password combination for x times

Comments are on the last page.

On this page
On this page

Installation

Fail2ban reads logfiles and acts based on their entries. For example, it can recognize when someone has entered a wrong password six times in two minutes and lock them out for half an hour.

# aptitude install fail2ban

Have Fail2ban start automatically at boot:

# systemctl enable fail2ban.service

Copy the conf file to a local differential file:

# cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

In /etc/fail2ban/jail.local set the following values:

backend = polling

(Alternatively set to auto but that made Fail2ban complain about pyinotify not being installed. That’s not problem: Fail2ban just tries all options but I don’t like complaints in my logfiles.)

mta = sendmail
destemail = your@email.here

Set action:
action_ = just ban
action_mw = ban and mail
action_mwl = ban and mail with whois report and relevant log lines
I suggest _mwl but change it to your needs.

action = %(action_mwl)s

Start Fail2ban:

# fail2ban-client start

If it was already running:

# service fail2ban restart

You should receive a mail notification at your specified e-mail address that Fail2ban has started.

Block random address spammers

Some spammers/phishers send mail to common e-mail addresses (john@; admin@) in the hope these addresses exist. If a sender sends mail to a bunch of non-existant addresses at the same time you may as well stop accepting mail from that sender.

In /etc/fail2ban/filter.d/postfix.conf add these lines under failrexeg:

reject: RCPT from (.*)\[<HOST>\]: 550 5.1.1
reject: RCPT from (.*)\[<HOST>\]: 450 4.7.1
reject: RCPT from (.*)\[<HOST>\]: 554 5.7.1

(Source: http://www.fail2ban.org/wiki/index.php/Postfix)

in /etc/fail2ban/jail.local under [Postfix] set

enabled = true

Users entering a wrong username/password combination for x times

This can indicate a brute-force attack. It’s up to you to decide if you want to use it. Personally I like to use it but set the limit higher than the default by adding

maxretry = 10

to the jail definition in /etc/fail2ban/jail.local.

For Roundcube:
In /etc/fail2ban/jail.local under [roundcube-auth] set

enabled = true

Also set

logpath = /var/log/roundcube/errors

In /etc/fail2ban/filter.d/roundcube-auth.conf set:

failregex = IMAP Error: (FAILED login|Login failed) for .*? from <HOST>

Likely someone will come up with a better regex to identify logon failures but for me this works.

For other imap clients:

In /etc/fail2ban/jail.local under [dovecot] set

enabled = true

Afterwards do

# service fail2ban reload

To manually unban a client do

# fail2ban-client set owncloud unbanip 192.168.1.2

To manually check the Fail2ban’s ownCloud jail:

# fail2ban-client status roundcube-auth
# fail2ban-client status dovecot
# fail2ban-client status postfix

Installing a mailserver on Debian 8 – Part 9: Using mail with a remote IMAP client (i.e. Thunderbird)

How to install a complete mailserver on Debian 8, featuring Postfix, Dovecot, MySQL, Spamassassin, ClamAV, Roundcube and Fail2ban.

~ the howto that actually works ~

Part 1: Introduction
Part 2: Preparations: Apache, Let’s Encrypt, MySQL and phpMyAdmin
Part 3: MTA: Postfix
Part 4: IMAP server: Dovecot
Part 5: Web interface: Roundcube
Part 6: Spam filtering: SpamAsasssin
Part 7: Antivirus: ClamAV and ClamSMTP
Part 8: Quota and other Roundcube settings
Part 9: Using mail with a remote IMAP client (i.e. Thunderbird)
Part 10: Counter brute-force attacks with Fail2ban
Part 11: Sources, config files, colouring and comments

On this page

Enable SSL
Incoming mail
Outgoing mail

Comments are on the last page.

On this page
On this page

To use IMAP you need to forward ports 993 (IMAP over TLS) and 465 (SMTP over TLS) from your router to your mailserver. In theory you can use any port you like but let’s stick to the rfc’s for now. I will use Thunderbird as an example client.

You've done it again, Gadget!
You’ve done it again, Gadget!

Enable SSL

In case your forgot read this bit about your website’s domain vs. your mail domain.

Turn on SSL first. (And by SSL I mean TLS you nitpicks.) We can use the Let’s Encrypt certificates we got for the webserver. In /etc/dovecot/conf.d/10-ssl.conf:

ssl = required
ssl_cert = </etc/letsencrypt/live/example.com/fullchain.pem
ssl_key = </etc/letsencrypt/live/example.com/privkey.pem 

Incoming mail

In /etc/dovecot/conf.d/10-master.conf we tell Dovecot about the protocol:

service imap-login {
  inet_listener imap {
  }
  inet_listener imaps {
    port = 993
    ssl = yes
  }
}

Be VERY careful in this document. Like we discussed before, if you mess up one # or { in this file you will be troubleshooting the rest of the day. See the part about setting up LMTP and SASL.

Do

# dovecot reload

and watch /var/log/mail.log for errors or warnings.

In Thunderbird use the following values:
Servername: example.com
Username: tinus@example.com
Port: 993
Connection type: SSL/TLS
Authentication method: Normal password

Thunderbird can be a bit stubborn. If you are sure your settings are correct and no obvious errors get logged but Thunderbird just won’t accept your settings just close it and try again. Also sometimes it says the settings are incorrect but will allow you to continue anyway and it just works.

Outgoing mail

In /etc/postfix/main.cf:

smtpd_tls_cert_file=/etc/letsencrypt/live/example.com/fullchain.pem
smtpd_tls_key_file=/etc/letsencrypt/live/example.com/privkey.pem
smtpd_use_tls=yes
smtpd_tls_auth_only = yes

In /etc/postfix/master.cf:

smtps     inet  n       -       -       -       -       smtpd
  -o syslog_name=postfix/smtps
  -o smtpd_tls_wrappermode=yes
  -o smtpd_sasl_auth_enable=yes

Note that the three option lines must start with at least one whitespace.

I suggest you open the TLS ports on your firewall but keep 143 (IMAP) closed. This prevents unencrypted IMAP sessions being set up in case of a configuration error.

# postfix reload

In Thunderbird:
Servername: example.com
Port: 465
Connection encryption: SSL/TLS
Authentication method: Normal password
Username: tinus@example.com

Mail settings in Thunderbird
Mail settings in Thunderbird

Installing a mailserver on Debian 8 – Part 7: Antivirus: clamav

How to install a complete mailserver on Debian 8, featuring Postfix, Dovecot, MySQL, Spamassassin, ClamAV, Roundcube and Fail2ban.

~ the howto that actually works ~

Part 1: Introduction
Part 2: Preparations: Apache, Let’s Encrypt, MySQL and phpMyAdmin
Part 3: MTA: Postfix
Part 4: IMAP server: Dovecot
Part 5: Web interface: Roundcube
Part 6: Spam filtering: SpamAsasssin
Part 7: Antivirus: ClamAV and ClamSMTP
Part 8: Quota and other Roundcube settings
Part 9: Using mail with a remote IMAP client (i.e. Thunderbird)
Part 10: Counter brute-force attacks with Fail2ban
Part 11: Sources, config files, colouring and comments

On this page

Installing
Automatically updating ClamAV
Testing the virus filter

Comments are on the last page.

On this page
On this page

Installing

ClamAV is a open source antivirus tool. It comes with Freshclam, which handles definition updates. Clamsmtp is the part that allows Clam to scan smtp traffic.

Installation:

# aptitude install clamav-daemon clamav clamsmtp

The clamsmtp manfile suggests the following. Change the clamsmtp folders ownership to clamav:

# chown -R clamav:clamav /var/spool/clamsmtp/
# chown -R clamav:clamav /var/run/clamsmtp/

in /etc/clamsmtpd.conf:

User: clamav

Restart the service:

# service clamsmtp restart

In /etc/postfix/main.cf we need to tell Postfix about the virus scanner. Add these lines:

# Virusscanner
content_filter = scan:127.0.0.1:10026
receive_override_options = no_address_mappings

in /etc/postfix/master.cf add:

# Antivirus
scan      unix  -       -       n       -       16      smtp
        -o smtp_send_xforward_command=yes

# For injecting mail back into postfix from the filter
127.0.0.1:10025 inet  n -       n       -       16      smtpd
        -o content_filter=
        -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks
        -o smtpd_helo_restrictions=
        -o smtpd_client_restrictions=
        -o smtpd_sender_restrictions=
        -o smtpd_recipient_restrictions=permit_mynetworks,reject
        -o mynetworks_style=host
        -o smtpd_authorized_xforward_hosts=127.0.0.0/8

More info: http://thewalter.net/stef/software/clamsmtp/postfix.html

As always make sure that the lines starting with -o begin with one or more whitespaces.

# postfix reload

Start ClamAV:

# service clamav-daemon start

Automatically updating ClamAV

ClamAV will update automatically. Initially you should keep an eye on /var/log/clamav/freshclam.log for errors.

In /etc/clamav/freshclam.conf change

DatabaseMirror db.local.clamav.net

to

DatabaseMirror db.XX.clamav.net

Restart ClamAV:

# service clamav-daemon start

Change XX to your country code (NL for The Netherlands, BE for Belgium, and so on). I vaguely remember that if you get this value right you are allowed to poll for updates every 15 minutes instead of 60 but I’m not sure ClamAV still do that.

Testing the virus filter

Testing antivirus is not easy. It is, in a sense, because you can just send the EICAR test virus to your server, only you can’t because no mailserver will allow you to send it. Even if you found one you can’t save the EICAR file locally because your local antivirus will delete the file. There are webservices that send EICAR files to your address but most are obsolete.

We’ll send an EICAR test virus from within our server using Mutt. Mutt is a versatile but lightweight e-mail client installed by default on Debian 8.

Do

# tail -f /var/log/mail.log | grep -i clam

and

# tail -f /var/log/clamav/clamav.log

to keep an eye on Clam’s activities, especially regarding the EICAR test file.

Fire up a new SSH session into your server and this time don’t su to root.
Download the EICAR test file:

$ wget http://www.eicar.org/download/eicar.com

Start Mutt:

$ mutt

Mutt will ask you if you want to create a spool file. That’s ok. In Mutt press m to start a new mail.
To: tinus@example.com
Subject: 01 – test
Now the Nano text editor is started. Type something in the body of the mail; it doesn’t matter what. Press Ctrl + X to stop editing.
Press Y to save the mail
Press Enter to confirm the filename.
Press y to send the mail. Mutt should now say the mail was sent.

/var/log/clamav/clamav.log should show no new entries and /var/log/mail.log | grep -i clam should show “from=vorkbaard@example.com, to=tinus@example.com, status=CLEAN”

Verify you have received the mail in Roundcube. Then start over: in Mutt, press m to start a new mail.
To: tinus@example.com
Subject: 02 – test
Type something in the body.
Ctrl + X to stop editing
Y to save
Enter to confirm the filename
Press a to add an attachment
type: eicar.com and press Enter (if the file wasn’t found use an absolute path).
eicar.com is now added as attachment
Press y to send the mail. Mutt should confirm the mail was sent.

EICAR is noted in the log files.
EICAR is noted in the log files.

/var/log/clamav/clamav.log should log something like

Tue Mar 22 15:27:47 2016 -> /var/spool/clamsmtp/clamsmtpd.N3lkUG: Eicar-Test-Signature(c88982d8fd9fe8013389c4f801a237b6:851) FOUND

and /var/log/mail.log | grep -i clam should show:

from=vorkbaard@example.com, to=tinus@example.com, status=VIRUS:Eicar-Test-Signature

Installing a mailserver on Debian 8 – Part 4: IMAP server: Dovecot

How to install a complete mailserver on Debian 8, featuring Postfix, Dovecot, MySQL, Spamassassin, ClamAV, Roundcube and Fail2ban.

~ the howto that actually works ~

Part 1: Introduction
Part 2: Preparations: Apache, Let’s Encrypt, MySQL and phpMyAdmin
Part 3: MTA: Postfix
Part 4: IMAP server: Dovecot
Part 5: Web interface: Roundcube
Part 6: Spam filtering: SpamAsasssin
Part 7: Antivirus: ClamAV and ClamSMTP
Part 8: Quota and other Roundcube settings
Part 9: Using mail with a remote IMAP client (i.e. Thunderbird)
Part 10: Counter brute-force attacks with Fail2ban
Part 11: Sources, config files, colouring and comments

On this page

Installing
Troubleshooting
Set mail location
Enable tranfer of mail delivery from Postfix to Dovecot
Have Dovecot look up accounts in MySQL rather than PAM

Comments are on the last page.

On this page
On this page

Installing

Install Dovecot with

# aptitude install dovecot-core dovecot-imapd dovecot-lmtpd dovecot-managesieved dovecot-mysql dovecot-sieve

Dovecot is configured modularly with files in /etc/dovecot/conf.d/ being included from /etc/dovecof/dovecof.conf.

Dovecot file structure
Dovecot file structure

Troubleshooting

Keep an eye on the current configuration with doveconf -n, which shows any settings with non-default values:

# watch -d doveconf -n

This is optional but will help troubleshooting.

First we’ll enable a lot of logging options for Dovecot. You can turn most of them off once everything works but this too will greatly ease your troubleshooting effort.

Open /etc/dovecot/conf.d/10-logging.conf. Do read the commentary in that file and any other configuration file you stick your fingers in.
Uncomment/change:

syslog_facility = mail
auth_verbose = yes
auth_verbose_passwords = yes
auth_debug = yes
auth_debug_passwords = yes
mail_debug = yes
verbose_ssl = yes

After saving do

# doveconf reload

Set mail location

In /etc/dovecot/conf.d/10-mail.conf edit:

mail_location = maildir:/var/mail/vmail/%d/%n

This will set the mail storage format to maildir and store all mail under /var/mail/vmail/domainname/username/, e.g. /var/mail/vmail/example.com/tinus/.

mail_privileged_group = vmail

This is the group used for creating mailbox directories

Reload the config with

# doveconf reload

Enable tranfer of mail delivery from Postfix to Dovecot

Unfortunately we can’t separate lmtp from sasl. Sasl is sort of a security add-on to lmtp. While you definitely should use it it would have been nice to be able to set it up (and thus test) it separately. So we’re doing both at once.

Normally Postfix would take care of the delivery of mail to users’ mailboxes. We’re now going to use lmtp to transfer that responsibility to Dovecot. From Wikipedia: “The Local Mail Transfer Protocol (LMTP) is a derivative of ESMTP, the extension of the Simple Mail Transfer Protocol (SMTP). (…) LMTP is designed as an alternative to normal SMTP for situations where the receiving side does not have a mail queue ” It’s mail transport between Postfix and Dovecot.

SASL is a method for allowing Postfix to use Dovecot’s authentication mechanisms. From Wikipedia: “Simple Authentication and Security Layer (SASL) is a framework for authentication and data security in Internet protocols. ” We need to tell Dovecot and Postfix to only accept each other’s (and mail clients’) mail to prevent unauthorized mails from being sent.

In /etc/postfix/main.cf add:

virtual_transport = lmtp:unix:private/dovecot-lmtp
smtpd_sasl_auth_enable = yes
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth

The private/dovecot-lmtp directory is relative to /var/spool/postfix/.
virtual_transport tells Dovecot to use the transport protocol lmtp.

/etc/postfix/master.cf describes who is allowed to use sasl:

smtp      inet  n       -       -       -       -       smtpd
submission inet n       -       -       -       -       smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=encrypt
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject

lmtp      unix  -       -       -       -       -       lmtp

Note: the -o lines must start with at least one empty space. Master.cf consists of ‘logical lines’ which can span multiple real lines. A real line is ended by a line ending; a logical line is ended after the last continuation’s line ending. A continuation is defined by one or more whitespaces at the start. It’s easiest to show this with an example:

One real line:

submission inet n       -       -       -       -       smtpd

One logical line:

submission inet n       -       -       -       -       smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=encrypt

So be careful to start continuation lines with at least one space and also the reverse: for example when uncommenting the first of a logical line be careful not to leave a whitespace in front of it or the line is seen as a continuation of the previous line resulting in all sorts of confusion.

In /etc/dovecot/conf.d/10-auth.conf:

auth_mechanisms = plain login

This will allow clients to log in with a password.

Open /etc/dovecot/conf.d/10-master.conf. Pay close attention; I always get this wrong, resulting in “no SASL methods available” errors. Use doveconf -n to check for more verbose error messages. If things break there is a good chance you removed or added a hash sign somewhere which caused a clause not to close, like this:

service blah {
	option {    <----------- The error is here...
		option 1 {
		option 2 {
			key_one = value_one
			key_two = value_two
		}
		option 3 {
		}
	}
service doh {       <----------- ...but the log file may mention this line.
}

The above code contains six opening and four closing brackets. Not good.

The whole file, while well intended, is easy to screw up. Because a missing closing bracket on a certain line is not formally an error the error you will get is about “service doh” not being a valid option.

So before you start making changes to this file create a backup copy first and very carefully watch and count opening and closing brackets.

/etc/dovecot/conf.d/10-master.conf should contain the following lines:

service lmtp { 
	unix_listener /var/spool/postfix/private/dovecot-lmtp {
		mode = 0600
		user = postfix 
		group = postfix 
		} 
	} 

service auth { 
	unix_listener /var/spool/postfix/private/auth {
		mode =0666
		user = postfix 
		group = postfix
		} 
	}

Comment out:

  #unix_listener auth-userdb {
    #mode = 0666
    #user =
    #group =
  #}

Check with

# doveconf -n

for error messages before you continue.

If /var/log/mail.log says “no SASL methods available” you probably made an error in /etc/dovecot/conf.d/10-master.conf.

Have Dovecot look up accounts in MySQL rather than PAM

In /etc/dovecot/conf.d/10-auth.conf comment out !include auth-system.conf.ext and uncomment !include auth-sql.conf.ext:

#!include auth-system.conf.ext
!include auth-sql.conf.ext

/etc/dovecot/dovecot-sql.conf.ext defines how Dovecot can find its way around in the database:

driver = mysql
connect = host=127.0.0.1 dbname=postfix user=mailman password=P@ssw0rd
default_pass_scheme = SHA512-CRYPT
password_query = \
  SELECT email as username, pwd AS password \
  FROM addresses WHERE email = '%u'
user_query = \
   SELECT 5000 AS uid, 5000 as gid, email, \
   '/var/mail/vmail/%d/%n' AS home \
   FROM addresses WHERE email = '%u'
iterate_query = SELECT email AS user FROM addresses

If you read the dovecot-sql.conf.ext commentary you may find that you can use static user queries. With static user queries all users lookups return the same uid and gid, which is what we need because of the dedicated account we use for permissions on the mail. So why pull it from the database with a query, you may ask. Because this is a so called dynamic user query. Dynamic user queries are necessary if you want to be able to use the iteration query. (You want to use the iteration query.)

The iterate query is necessary for certain Dovecot commands like searching for mail for management purposes, such as automatically deleting spam older than thirty days or compacting mailboxes.

Let me break down the user query because we’re doing a not very common thing here.
SELECT 5000 AS uid, 5000 as gid means: get the value for ‘uid’ from the table but pretend it is 5000. Do the same for the gid. (In fact we have no uid nor a gid field in the database.)
email: well we definitely want to pull the e-mail address from the database.
‘/var/mail/vmail/%d/%n’ AS home: get the value for ‘home’ from the table but pretend it is ‘/var/mail/vmail//.
Dovecot replaces %d, %n and %u on the fly. For more info on this fun feature check out http://wiki.dovecot.org/DovecotFeatures#User.2BAC8-Password_Databases.

Do

# dovecot reload && postfix reload

… while keeping an eye on /var/log/mail.log for possible error messages or warnings.

Test if you can still receive and send mail.

Error: warning: connect to mysql server 127.0.0.1: Access denied for user ‘mailman’@’localhost’ (using password: YES)

Make sure you are using the same host, username and password in MySQL, /etc/postfix/mysql_virtual_mailbox_maps.cf and /etc/dovecot/dovecot-sql.conf.ext.

While I was reviewing this article I used P@ssword in one instance and P@ssw0rd in another. I ended up changing everything to P@ssw0rd. To change a user password in MySQL:

# mysql -u root -p
mysql> SET PASSWORD FOR 'mailman'@'127.0.0.1' = PASSWORD('P@ssw0rd');
mysql> quit

We’ll do more Dovecot configuration in the next part. Remember, this is a topic based series 😉

Installing a mailserver on Debian 8 – Part 3: MTA: Postfix

How to install a complete mailserver on Debian 8, featuring Postfix, Dovecot, MySQL, Spamassassin, ClamAV, Roundcube and Fail2ban.

~ the howto that actually works ~

Part 1: Introduction
Part 2: Preparations: Apache, Let’s Encrypt, MySQL and phpMyAdmin
Part 3: MTA: Postfix
Part 4: IMAP server: Dovecot
Part 5: Web interface: Roundcube
Part 6: Spam filtering: SpamAsasssin
Part 7: Antivirus: ClamAV and ClamSMTP
Part 8: Quota and other Roundcube settings
Part 9: Using mail with a remote IMAP client (i.e. Thunderbird)
Part 10: Counter brute-force attacks with Fail2ban
Part 11: Sources, config files, colouring and comments

On this page

Real vs. virtual users
Installation
Configuration
Sending mail
Receiving mail
Setting myhostname
Which domains to accept mail for
Which addresses to accept mail for
Change to maildir
Connect Postfix to the MySQL database
Message size limit

Comments are on the last page.

On this page
On this page

Postfix is the software that listens for incoming mail on port 25 and sends outgoing mail to other mailservers. It stores mail in mailboxes you define.

Real vs. virtual users

Postfix differentiates between ‘real’ and ‘virtual’ users (and real and virtual domains). Real users are entities (people) that have a user account on the server. Virtual users are entities that have only e-mail accounts, not necessarily server accounts.

Read more about virtual vs. real here: http://www.postfix.org/VIRTUAL_README.html

Installation

# aptitude install postfix postfix-mysql

Upon installation you will be replacing Debian’s default mailserver Exim4. Accept the installer’s solution to remove exim4 and related packages.

The following actions will resolve these dependencies:

     Remove the following packages:
1)     exim4
2)     exim4-base
3)     exim4-config
4)     exim4-daemon-light



Accept this solution? [Y/n/q/?] y

Postfix will ask you about your configuration. I’m going to install the ‘Internet site’ version: mail is sent and received directly using SMTP.

Install the Internet Site configuration
Install the Internet Site configuration

System mail name: example.com
If you’ll be setting up your mailserver for multiple domains just pick a main domain; it should be the domain that’s most likely to stay on your server the longest. In technical regards it doesn’t really matter.

Enter your domain name
Enter your domain name

Configuration

Postfix’s mail configuration files are /etc/postfix/main.cf and /etc/postfix/master.cf. Main.cf contains application settings while master.cf contains transport settings (which protocol to use, which port number, etc.).

Sending mail

You don’t need to be change anything to be able to send out mail. Keep on eye on your log file with

# tail -f /var/log/mail.log

and in another session send a test mail with:

# echo "This is a test mail" | mailx -vvv -s "test 01" your@email.here
Mail Delivery Status Report will be mailed to <root>.

Line 2 is the server’s reply if all is well. It is normal and does not indicate an error.

I like to number my test mails so I know which one I received if I need to test more than once. -s specifies the subject and -vvv sets verbosity to high so you can view everything that’s going on.
If the mail doesn’t arrive try sending to a non-Gmail account or an account on some other server you know will not block your mail for spam.

Receiving mail

Test if you can receive mail by sending a mail to vorkbaard@example.com (use your own username). Check /var/mail/vorkbaard (or whatever username you have) for growth. I did that by running

watch -d -n 1 du -cs /var/mail

in a tmux pane.
(Press Ctrl + C to stop.)

tmux with file size in the upper and log file in the lower pane
tmux with file size in the upper and log file in the lower pane

Once you proved your system can send and receive mail you can continue. If one or both do not work troubleshoot until it does. Check your router, firewall, DNS settings, the logfiles (/var/log/mail.log, /var/log/syslog) and mxtoolbox.com for pointers.

Only if sending and receiving works continue with the rest of this article. That way if something doesn’t work further on you will at least know where you don’t need to look. (Namely: your isp (it’s not blocking port 25); your router (port forwarding works); local firewalls (they’re not blocking mail access); etc.)
A succesful mail delivery ends in these logfile lines:

... Successful quit
... removed

Setting myhostname

The myhostname value in Postfix is read by mailservers receiving mail from your system. Your IP address should correspond to your fully qualified domain name (FQDN). If not some spamfilters will raise their eyebrows and add a point to your spam score.

To retrieve your FQDN send a ping to your registered hostname:

# ping mail.example.com
PING mail.example.com (12.34.56.78) 56(84) bytes of data.
64 bytes from 12.34.56.78.dynamic.upc.nl (12.34.56.78): icmp_seq=1 ttl=64 time=0.172 ms

In my case it was 12.34.56.78.dynamic.upc.nl. Enter that as your myhostname value in /etc/postfix/main.cf.

To follow the process you can enter your domain name in mxtoolbox.com and have them perform an SMTP test on your host. Notice that it says ‘Reverse DNS does not match SMTP banner’. After you changed the myhostname do

# postfix reload

and run the test again. It should now say: ‘OK – 12.34.56.78 resolves to 12.34.56.78.dynamic.upc.nl’

On a side note: my ISP gives me semi-static IP address. That is, it’s dynamically assigned but hasn’t changed in years. Because I am on a consumer line using a dynamic address from an ISP pool Spamhaus listed me on their PBL. Spamhaus’s PBL is a list of (among other things) dynamic public IP addresses that have no business running mail servers. However my ISP has allowed me to run a mail server, I don’t run an open relay and take care not to (accidentally) send out spam from my network so I used Spamhauses self-delisting tool to remove my public ip address from the PBL.

SORBS is refusing to remove me from their DUHL list of dynamic IP ranges.

You will need to keep an eye out for things like this when running a mailserver on a consumer line.

Which domains to accept mail for

Open up /etc/postfix/main.cf and look at the mydestination value.

mydestination = example.com, mail.localdomain, localhost.localdomain, localhost

This value tells Postfix which domains to accept mail for. The interesting part of course is ‘example.com’. If you’re going to host more domains add them here. You can do it in a separate file, you can do it in a database. I’m doing it in main.cf.

Which addresses to accept mail for

As discussed, Postfix can handle two types of users: virtual users and real users. Root is a real user and in my case vorkbaard is because they are user accounts on my server. However most the people who are going to get an e-mail account on my server will never log on to the server to work with a local user account. No home folders on the server, no bash preferences, etc. So for mail we’re going to use virtual users.

With real users, the question which user should own the mail had a simple answer: the user. With virtual users this is a bit different. Since no real user exists, who should own the mail? Postfix provides two options: 1 – create virtual uid’s and gid’s to go along with the virtual user accounts or 2 – use one dedicated uid and gid for all users.

Since the virtual users won’t be logging on to the server there’s no harm in them not being the owner of their mail. Option 2 is easier and not less safe so I’ll go with option 2.

More on virtual mailboxes: http://www.postfix.org/VIRTUAL_README.html#virtual_mailbox

in main.cf add:

virtual_mailbox_domains = example.com

This tells Postfix which domains to listen for mail to. This replaces the entry in mydestination so remove example.com from the mydestination value. If you don’t /var/log/mail.log will inform you that you should not list domains both in mydestination and in virtual_mailbox_domains.

virtual_mailbox_base = /var/mail/vmail

This is the folder where mail will be stored.

virtual_mailbox_maps = hash:/etc/postfix/virtual_mailbox_maps.cf

virtual_mailbox_maps tells Postfix where to look for virtual e-mail accounts.

Add information about the owner of the mailboxes:

virtual_gid_maps = static:5000
virtual_uid_maps = static:5000
virtual_minimum_uid = 5000

virtual_minimum_uid is used for setups where you will be using separate users for each mailbox. It prevents receiving mail for system accounts. We will use one owner for all mail and its uid and gid will be 5000 so the virtual_mininum_uid can be 5000 as well.

Let’s create the owner:

# useradd -d /var/mail -U -u 5000 vmail

This sets the user’s home folder to /var/mail, creates a group with the same name as the user, adds the user to it and sets the user’s uid to 5000. The username is vmail.

Note that ‘vmail’ is not a reserved name. It stands for ‘virtual mail’ and is often used in articles explaining how to set up mailservers on Linux. You can choose any name you want and store the mail in any directory you want as long as you have the corresponding uid and gid values in main.cf and deal out the right permissions on the directories you’re using.

Since we’re going to store mail in /var/mail/vmail we should create the folder and give user vmail access to it. Create a folder under /var/mail/vmail for each domain you’ll be receiving mail for.

# mkdir -p /var/mail/vmail/example.com
# chown -R vmail:vmail /var/mail/vmail 

create /etc/postfix/virtual_mailbox_maps.cf and add:

tinus@example.com example.com/tinus

(Tinus is Tinus de Tester, my default test user.)

The format is:

email@example.com  	where_to_store_it

The storage location is relative to virtual_mailbox_base. So mail for tinus@example.com will be stored in /var/mail/vmail/example.com/tinus.

User Tinus’ would be stored in /var/mail/vmail/example.com/tinus.

After the changes do

# postfix reload
# postmap /etc/postfix/virtual_mailbox_maps.cf

That last command will create a file /etc/postfix/virtual_mailbox_maps.cf.db which is faster for Postfix to read.

Now send a mail to tinus@example.com and see if a file is created under /var/mail/vmail/example.com/. If not check your logfile, any non-delivery reports, and so on.

My mailserver (/var/log/mail.log) spat this at me:

Recipient address rejected: User unknown in local recipient table

Hmm… The local recipient table? I’m not doing local recipients, remember? Turned out I forgot to set virtual_mailbox_domains.

Only continue if mail gets delivered (=the file /var/mail/vmail/example.com/tinus is created or increases in size when you send a mail to tinus@example.com).

At this point you have a working mail system which is able to send and receive mail. Anything outside your server will not be the cause of mail not going out or coming in (safe for obvious exceptions such as a failing internet connection).

Change to maildir

By default Postfix uses the mbox format to store mail. In short: mbox stores all mail in one big file. The advantage is it uses just one inode (which more or less means it uses one file) and is easy to back up; the drawback is it depends on lockfiles and doesn’t allow access from more than one source concurrently, and is relatively easy corrupted.

We’ll be changing to the maildir format which uses one file per e-mail item. Its advantages and drawbacks are inverted relative to mbox.

First delete /var/mail/vmail/example.com because the system cannot create a directory if a file exists with the same name in the same parent directory. Also delete its folder; the domain folder should be created automatically.

# rm -rf /var/mail/vmail/example.com

A lot of howtos will tell you to add home_mailbox = Maildir/ and mailbox_command = to your main.cf. However since we’re using virtual users we only need to change one thing and it is not in main.cf: in /etc/postfix/virtual_mailbox_maps add forward slashes to the locations. Change

tinus@example.com example.com/tinus

to

tinus@example.com example.com/tinus/

Do

# postmap /etc/postfix/virtual_mailbox_maps.cf

to rehash the mailbox database.

Verify mail can still be sent and received before you continue. A mailbox structure should have been created under /var/mail/vmail/example.com/tinus/.

Why go to the trouble of setting up Postfix, changing to virtual mailboxes, changing to maildir if we’re going to use a database to handle the mailboxes? Because this way you will know for certain that the permission structure is in order and mail can be sent and received. Again: if this works and you will encounter a problem in the next section you will know where not to look. So, easier troubleshooting.

Connect Postfix to the MySQL database

Instead of using a file to manage the list of users you can have Postfix checking your database. Why? My main reason was because Roundcube, the web interface for mail I am using, has a plugin that allows users to change their IMAP passwords. Which is pretty essential for any mail system if you ask me. There’s also a plugin for PAM but we’re not using real users.

Storing users in a database has other advantages too, the main one that comes to mind is custom fields to select on. To illustrate this I added a boolean field called ‘active’. If set to 0 the account is suspended, if set to 1 it’s active.

Start by deleting /etc/postfix/virtual_mailbox_maps.cf and /etc/postfix/virtual_mailbox_maps.cf.db.
Open main.cf and change

virtual_mailbox_maps = hash:/etc/postfix/virtual_mailbox_maps.cf

to

virtual_mailbox_maps = mysql:/etc/postfix/mysql_virtual_mailbox_maps.cf

The filename mysql_virtual_mailbox_maps.cf is arbitrary however it seems like a good idea to make it somewhat descriptive of its purpose. It is a common name for this kind of file.

# postfix reload

We need to tell Postfix how to connect to the database. Create /etc/postfix/mysql_virtual_mailbox_maps.cf and add:

user = mailman
password = P@ssw0rd
hosts = 127.0.0.1
dbname = postfix
query = SELECT 1 FROM addresses WHERE email = '%s'

Postfix replaces %s by the “input key” (in our case the e-mail address).

More info: http://www.postfix.org/mysql_table.5.html

Enter user Tinus in the database:

# mysql -u root -p
mysql> INSERT INTO `postfix`.`addresses` (`id`, `active`, `email`, `pwd`) VALUES ('1', '1', 'tinus@example.com', '');
mysql> quit

or use phpMyAdmin to do it. You can leave the password for now, we’ll get to that later.

See if it works:

# postmap -q tinus@example.com mysql:/etc/postfix/mysql_virtual_mailbox_maps.cf

This should return 1 for existing addresses and nothing for non-existant addresses.

If it doesn’t work, check these things:
– Did you enter the real e-mail address in the database or did you copy the example.com address from this article?
– Is the mysql service running? (By default it should be.)
– Did you type the correct password in /etc/postfix/mysql_virtual_mailbox_maps.cf and the database?

Now if you send an e-mail to tinus@example.com Postfix will create a file called 1 under /var/mail/vmail. Why? Because we no longer define where to store mail. We’re going to do that with Dovecot via lmtp – the local mail transport protocol. Make sure the file /var/mail/vmail/1 is created though!

Troubleshooting:
– Check your /var/log/mail.log
– verify all files mentioned in /etc/postfix/main.cf actually exist. I had a log file entry “ warning: mysql:/etc/postfix/virtual_mailbox_maps.cf lookup error for “tinus@example.com””; it turned out in main.cf I had virtual_mailbox_maps = mysql:/etc/postfix/virtual_mailbox_maps.cf instead of virtual_mailbox_maps = mysql:/etc/postfix/mysql_virtual_mailbox_maps.cf. See the difference?

Message size limit

The default maximum message size Postfix allows is 10MB. Is you need to change it open /etc/postfix/main.cf and enter

message_size_limit=20480000

for 20MB for example. The value is in bytes. More info: http://www.postfix.org/postconf.5.html#message_size_limit

Installing a mailserver on Debian 8 – Part 2: Preparations: Apache, Let’s Encrypt, MySQL and phpMyAdmin

How to install a complete mailserver on Debian 8, featuring Postfix, Dovecot, MySQL, Spamassassin, ClamAV, Roundcube and Fail2ban.

~ the howto that actually works ~

Part 1: Introduction
Part 2: Preparations: Apache, Let’s Encrypt, MySQL and phpMyAdmin
Part 3: MTA: Postfix
Part 4: IMAP server: Dovecot
Part 5: Web interface: Roundcube
Part 6: Spam filtering: SpamAsasssin
Part 7: Antivirus: ClamAV and ClamSMTP
Part 8: Quota and other Roundcube settings
Part 9: Using mail with a remote IMAP client (i.e. Thunderbird)
Part 10: Counter brute-force attacks with Fail2ban
Part 11: Sources, config files, colouring and comments

On this page

Setting up Apache
Let’s Encrypt
Install MySQL
Optional: setting up PhpMyAdmin for database administration
Creating the user database
Creating the databases from the command line
Creating the databases from phpMyAdmin
Setting permissions from the command line
Setting permissions from phpMyAdmin

Comments are on the last page

On this page
On this page

Setting up Apache

If you haven’t installed Apache and PHP yet do so now by running

# tasksel

and selecting Web server. If you want to do it manually install Apache2 and PHP.

Creating a website
Open /etc/apache2/sites-available/000-default.conf and change

#ServerName www.example.com

to

ServerName example.com

Reload Apache:

# service apache2 reload

Verify your site is reachable from the internet on http://example.com. You should see the Apache2 Debian Default Page. If not, check /var/log/apache/error.log.

Apache2 Debian Default page
Apache2 Debian Default page

Let’s Encrypt

You can purchase commercial certificates. If you run a business you should at least consider it. Alternatively use self-signed certificates – those would be most appropriate if you’re running everything on an intranet and/or have your own corporate CA.

Thanks to the Let’s Encrypt initiative we can now have free of charge certificates signed by a trusted third party and that’s what I’ll describe here.

Your mail domain vs. your website’s domain
It is important to understand that your website’s domain does not need to equal your mail domain. It is perfectly valid to host mail for aaa.com, bbb.com with a web interface on ccc.com. The SSL certificates you would be using would be those for ccc.com and in configuration files (e.g. /etc/dovecot/conf.d/10-ssl.conf) you would need to enter

ssl_cert = </etc/letsencrypt/live/ccc.com/fullchain.pem
ssl_key = </etc/letsencrypt/live/ccc.com/privkey.pem

The certificates you are receiving from Let’s Encrypt need to reflect your website’s domain, which may be identical to your mail domain but it certainly is not necessary.

Backports
Note that at the moment you need to install from the backports. Aptitude will give you a bunch of options for resolving dependencies. Choose the one that does not leave any dependency unmet – for me that was the third option but that is likely to change per system and over time. I needed to let it install 8 packages.

** NOTE 2016-06-07: Jessie’s backports currently holds broken dependencies for Letsencrypt. If you are unable to install Letsencrypt using the method described below, do:

# aptitude install certbot python-certbot-apache

END OF NOTE ***

Install Let’s Encrypt:

# aptitude install letsencrypt python-letsencrypt-apache

The following actions will resolve these dependencies:

     Keep the following packages at their current version:
1)     letsencrypt [Not Installed]
2)     python-acme [Not Installed]
3)     python-letsencrypt [Not Installed]
4)     python-letsencrypt-apache [Not Installed]



Accept this solution? [Y/n/q/?] n

The following actions will resolve these dependencies:

     Install the following packages:
1)     python-cffi-backend [1.4.2-2~bpo8+1 (jessie-backports)]
2)     python-cryptography [1.1.1-1~bpo8+1 (jessie-backports)]
3)     python-enum34 [1.0.3-1 (stable)]
4)     python-idna [2.0-3~bpo8+1 (jessie-backports)]
5)     python-ipaddress [1.0.16-1~bpo8+1 (jessie-backports)]
6)     python-openssl [0.15.1-2~bpo8+1 (jessie-backports)]
7)     python-pyasn1 [0.1.9-1~bpo8+1 (jessie-backports)]

     Keep the following packages at their current version:
8)     python-cffi [Not Installed]



Accept this solution? [Y/n/q/?] n

The following actions will resolve these dependencies:

     Install the following packages:
1)     python-cffi [1.4.2-2~bpo8+1 (jessie-backports)]
2)     python-cffi-backend [1.4.2-2~bpo8+1 (jessie-backports)]
3)     python-cryptography [1.1.1-1~bpo8+1 (jessie-backports)]
4)     python-enum34 [1.0.3-1 (stable)]
5)     python-idna [2.0-3~bpo8+1 (jessie-backports)]
6)     python-ipaddress [1.0.16-1~bpo8+1 (jessie-backports)]
7)     python-openssl [0.15.1-2~bpo8+1 (jessie-backports)]
8)     python-pyasn1 [0.1.9-1~bpo8+1 (jessie-backports)]



Accept this solution? [Y/n/q/?] y

Enable encryption on your site:

# letsencrypt run

mailsvr005-1

The installer asks for your e-mail address and wether you would like to redirect all HTTP requests to HTTPS. Since we’ll be using the site as a web interface for a mailserver I suggest you do.

Redirect HTTP to HTTPS
Redirect HTTP to HTTPS

Verify your site is now available at https://example.com.

Site accessible via HTTPS
Site accessible via HTTPS

Schedule the Let’s Encrypt update to run every week:

# crontab -e

Enter this line:

@monthly /usr/bin/letsencrypt renew –-agree-tos

I recommend you read up on Let’s Encrypt renewals here: https://letsencrypt.org/getting-started/ and in the letsencrypt manpage because Let’s Encrypt is still in development.

Install MySQL

# aptitude install mysql-client mysql-server

Remember the root password you enter.

Optional: setting up PhpMyAdmin for database administration

PhpMyAdmin is a webbased database administration tool. Its installation is optional but I like it because it gives a good overview of your databasian situation.

# aptitude install phpmyadmin

Choose the Apache webserver when asked, unless you know you’re running a different webserver.

Choose Apache
Choose Apache

Have the installer create a database for you.

mailsvr008

Use the root password you entered when installing mysql-server.

Enter MySQL's root password
Enter MySQL’s root password

Enter an application password for phpMyAdmin when the installer asks for it. You do not need this password afterwards so a random password is ok. Just leave the field empty.

Application password
Application password

After installation you can find phpMyAdmin at https://example.com/phpmyadmin. You can log in with user root and the MySQL root password.

phpMyAdmin
phpMyAdmin

For extra security, deny access to phpMyAdmin from outside your own network by editing /etc/phpmyadmin/apache.conf. Below <Directory /usr/share/phpmyadmin> insert these lines:

Order deny,allow
# Allow from 192.168.1.1/24
# Allow from 192.168.1.1/255.255.255.0
# Allow from 12.34.56.78
# Allow from 12.34.56.78 87.65.43.21
Deny from all

Uncomment and edit any appropriate Allow lines. Lines 1 and 2 are equal, only the notation differs. Afterwards reload Apache to effectuate the settings:

# service apache2 reload

Always verify your settings take: go to https://example.com/phpmyadmin from an address not listed as allowed and make sure you can’t get in.

Creating the user database

There is a near infinite number of ways to set up your database. I’ll be working with two tables: one for virtual users and one for virtual aliases. (More on virtual users later.)

A couple of notes on your database design.

  • Choose UTF8 as the password collation. This will allow weird ascii characters to be used.
  • Don’t worry if phpMyAdmin says your collation is Swedish – this is a phpMyAdmin quirk that occurs because it adds up all the collations and then finds the one that corresponds. Just ignore it.
  • Email address length can be bigger but you’ve got to draw the line somewhere and I think fifty characters is enough.
  • varchar(106) for the password field allows it to be encrypted with SHA512.
  • “password” is a protected word in MySQL so it’s best not to confuse things unnecessarily and avoid it. That’s why I named the table ‘pwd’.
  • In principle the e-mail address is unique so it could serve as a primary key. However e-mail addresses may change and besides it is best practice to have a dedicated primary key.
  • If you’re going to use this for a production server think very carefully about your database setup. A small choice now can have a huge effect later.

Option 1: from the command line
Log into MySQL:

# mysql -u root -p

For the next lines be careful to use backticks ( ` ) where needed.

Creating the databases from the command line

mysql> CREATE DATABASE postfix;
mysql> USE postfix;

Create the user table:

mysql> CREATE TABLE `addresses` (
`id` int NOT NULL AUTO_INCREMENT,
  `active` tinyint(1) NOT NULL,
  `email` varchar(50) NOT NULL,
  `pwd` varchar(106) NOT NULL,
  PRIMARY KEY (`id`),
  UNIQUE (`email`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
quit

Create the aliases table:

CREATE TABLE `aliases` (
`id` int NOT NULL AUTO_INCREMENT,
  `source` varchar(50) NOT NULL,
  `target` varchar(50) NOT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
mysql> quit

Creating the databases from phpMyAdmin

Select the Databases tab and create a database called postfix. Set the collation to utf8_general_ci.

mailsvr010

Select the new table.

mailsvr010b

Create a table called addresses with 4 columns.

mailsvr011

Create these colums:
name: id; type: INT; Index: PRIMARY, A_I (=auto-increment)
name: active; type: TINYINT; length: 1
name: email; type: VARCHAR; length: 50; index: UNIQUE
name: pwd; type: VARCHAR; length: 106

mailsvr012

Verify the settings by viewing the new table’s structure.

View the table's structure
View the table’s structure

Note the Primary and Unique properties.

mailsvr012c

Create a second table.

Create a new table
Create a new table

Create these columns:

Table name: aliases, 3 columns
Name: id; type: INT; Index: PRIMARY
Name: source; type: VARCHAR; length: 50
Name: target; type: VARCHAR; length: 50

Create aliases table
Create aliases table

View the new table’s structure.

View the aliases table's structure
View the aliases table’s structure

Verify all settings are correct.

mailsvr012g

Setting permissions from the command line

CREATE USER 'mailman'@'127.0.0.1' IDENTIFIED BY '***';
GRANT SELECT ON `postfix`.`addresses` TO 'mailman'@'127.0.0.1';
GRANT SELECT ON `postfix`.`aliases` TO 'mailman'@'127.0.0.1';
quit

Setting permissions from phpMyAdmin

Click on the home icon in the top left corner of phpMyAdmin to return to the start page.

The Home icon
The Home icon

Select the Users tab.

The Users tab
The Users tab

Click Add user.

mailsvr013

Create a user called mailman with host 127.0.0.1. For all values select ‘Use text field’. Remember the password (in these articles I’m using P@ssw0rd as an example). Then scroll down and press Go.

mailsvr014

From the users overview click the Edit Privileges link in mailman’s row.

mailsvr015

We don’t want to deal out global privileges; select Database.

mailsvr016

Select the postfix database.

mailsvr017

I like to explicitly name the tables. Select the addresses table.

Select the addresses table
Select the addresses table

Select all fields in the SELECT column. id is not really necessary so you can leave it out if you’re not using it.

mailsvr019

phpMyAdmin shows you the precise query after the execution which is rather educational.

mailsvr020

Do the same for the aliases table. Note that phpMyAdmin will present a slightly different look now because the postfix database is already selected.

mailsvr019b

mailsvr019c

mailsvr019d

mailsvr019e

mailsvr019f

Installing a mailserver on Debian 8 – Part 1: Introduction

How to install a complete mailserver on Debian 8, featuring Postfix, Dovecot, MySQL, Spamassassin, ClamAV, Roundcube and Fail2ban.

~ the howto that actually works ~

Part 1: Introduction
Part 2: Preparations: Apache, Let’s Encrypt, MySQL and phpMyAdmin
Part 3: MTA: Postfix
Part 4: IMAP server: Dovecot
Part 5: Web interface: Roundcube
Part 6: Spam filtering: SpamAsasssin
Part 7: Antivirus: ClamAV and ClamSMTP
Part 8: Quota and other Roundcube settings
Part 9: Using mail with a remote IMAP client (i.e. Thunderbird)
Part 10: Counter brute-force attacks with Fail2ban
Part 11: Sources, config files, colouring and comments

On this page

Target audience
Article structure: topic based layout
Requirements
Backports
Test domain
Port 25
Logging and troubleshooting

Target audience

These articles are intended for those who want to learn about mailservers on Linux. It covers applications, not hardware and operating system setup. If you’re going to implement a mailserver in a business environment read up on server hardening, backing up and restoring your setup and high availability techniques.

If you follow this article you will end up with a working mailserver and gain some insight in what you are doing, which you can build on to further increase your expertise.

The gist
The gist

Article structure: topic based layout

My instructions are based on topics, not software. A lot of howtos on Postfix and Dovecot will provide information based on files: here is file A, change this or that. Here is file B, comment out line number twelve. While that may or may not help you set up your mailserver this approach will a) not bring you much insight in what exactly you are doing and b) make troubleshooting a lot harder because mail not getting sent or received can be caused by any number of settings.

In this article series we will set up a basic mailsystem and test if it works. I’ll provide the necessary tools to troubleshoot the setup. Only if it works we’ll build further and after every step we stop, test and troubleshoot if necessary. If something breaks you will know exactly where to look and how to look for it.

Do read the documentation
If you want to understand the components – which you do – you should really read the reading suggestions. Man pages are a good place to start and most of the software we’ll be using has excellent online documentation. You don’t need to learn it all by heart. Reading documentation before installation will get you an impression of what the software is capable of.

Comments

Comments are on the last page of the article. I thought it would be good to see what others have asked before you post your own question.

Requirements

You need a domain name with access to its DNS settings. You’ll be out a fistfull of euro’s.

I assume you’re running on a freshly installed Debian 8 server with the LAMP stack installed.

There are several approaches to our goal but like we discussed, my approach is to build one component at a time rather than everything at once so you’ll know what you’re doing. Effectively this means you may need to edit the same configuration file more than once and even undo some changes while you’re progressing.

Backports

I’m using Debian 8.3 with all packages from the Debian repositories. At the moment of writing Roundcube is not available in Debian’s default repos but it is from the backports. I’m using the Debian packages because they’re well-maintained (for the software we’ll be using anyway) and because I value stability over newer features.

To enable backports on Debian 8 do:

# echo deb ftp://ftp.nl.debian.org/debian jessie-backports main contrib non-free > /etc/apt/sources.list.d/backports.list
# aptitude update

More on backports: http://backports.debian.org/Instructions/

Test domain

For this series I’m using the fictional domain example.com. Be sure to replace it with your own domain. Also make sure your DNS records are in order for your domain, especially your MX and SPF records. Missing SPF is not a technical problem but a misconfigured SPF record can be.

Where I use your@email.here you should use your own e-mail address. And don’t use the P@ssw0rd password I use in the examples…

Port 25

I’m installing on a consumer connection which I can because my ISP gives me a semi-static IP address. Some providers block outgoing traffic on port 25 by default, and with good reason. Most will open port 25 for outgoing traffic on your connection if you ask them politely.

To prevent spam from being sent from other hosts on my network I closed off port 25 for outgoing traffic on my firewall for all hosts except my mailserver. However that alone may not be enough. Spamhaus lists IP addresses that are infected with certain malware that may have nothing to do with e-mail, such as the Ponmocup trojan downloader. Spamfilters looking at Spamhaus will then mark mail from those addresses as spam so not only must your mailservice be in order, you must also make sure the rest of your network behaves.

Logging and troubleshooting

I like to run my sessions in tmux. Tmux allows running multiple terminal sessions in one SSH connection. The lower third of my screen runs the mail logfile and I keep an eye on the Postfix config with watch -d postconf -n in another pane of tmux. This setup allows for very easy troubleshooting. If you haven’t worked with tmux before now is a good time to start. Its learning curve will be very much worth your while.

To install it do

# aptitude install tmux

More on tmux: https://en.wikipedia.org/wiki/Tmux and

$ man tmux
tmux showing three panes
tmux showing three panes

Where to get clues for troubleshooting
Logfiles are your starting point for troubleshooting. If things don’t work, check your logs. By convention log files go into /var/log/ on Debian (based) systems. Many programs allow you to set a custom log file path so if you can’t find its logfile see if you can find its location in the program’s configuration files. If a logfile isn’t created at all see if the account that should create/add to it has permissions to do so.

I like to keep logfiles running while testing so I can see what’s going on. To do so do

# tail -f /var/log/mail.log

for example. You can stop it with Ctrl + C.

Furthermore Postfix can show you these things (from postconf’s man page):
postconf -p Show main.cf parameter settings. This is the default. [Use in conjunction with grep.]
postconf -n Show only configuration parameters that have explicit name=value settings in main.cf. Specify -nf to fold long lines for human readability

Dovecot has a similar feature (from doveconf’s man page):
doveconf -a Show all settings with their currently configured values.
doveconf -d Show the setting’s default value instead of the one currently configured.
doveconf -n Show only settings with non-default values.

Both postconf and doveconf have more options; read their respective man pages.

I wish you a lot of fun following this howto, and don’t forget to colour the colouring picture in the last installment!