HOWTO: Home Mail Server with TLS and non-Plain authenticatio

Anonymous · Oct 13, 2013

Abstract

This is the start of a How To on setting up a Home Mail Server that is meant to help to improve Citizens Privacy in e-mail communication. It will come in several chapters, each of which will occupy its own message on this thread.

Setting up your own mail server is quite involved, and you don't want to go into that if you are happy with the privacy declaration of your favourite mail hosting provider. If not, then your own home mail server could take out most of your e-mail communication (probably not all) from the radar of those many interested third parties.

Please don't expect the perfect privacy solution for all your communication needs. This does not exist by the way. A proper Home Mail Server installation will only help to prevent that your favorite Secret Service reads all your e-mails, they would have access only to some of them, and in the course of fine tuning your setup this portion could become less every day.

This How To is still a work in progress, and the chapters will be completed step-by-step by the end of this month.

This tread will start with an introduction to authentication mechanisms and password security, which is the foundation for the choices of software and configuration options made for the later chapters.

Table of Content

[POST=236392]Abstract[/POST]
1. Introduction

[POST=236393]2. Requirements[/POST]

2.1 The ISP must not block port 25
2.2 The Domain Name
2.3 Dynamic DNS Configuration

2.3.1 Checking Dynamic DNS
2.3.2 Creating a Dynamic DNS Request File and the Update Script
2.3.3 Utilizing the Update Script for Automatic Dynamic DNS Updating

[POST=236394]3. Installations[/POST]

3.1 Installation of OpenSSL from the ports, and re-building of all ports depending on OpenSSL
3.2 Installation of Dovecot IMAP Server + Dovecot SASL
3.3 Installation of Cyrus SASL2
3.4 Postfix Mail Server

[POST=236397]4. Configurations[/POST]

4.1 Creating a self-signed certificate chain
4.2 Configuration of Dovecot IMAP/POP3

4.2.1 The Virtual Users Store
[POST=236398]4.2.2 The Mail Directory[/POST]
4.2.3 The Dovecot Configuration File

4.3 Configuration of the Postfix Mail Transfer Agent

4.3.1 Relay Host Configuration
4.3.2 Local Recipients and Mail Aliases
4.3.3 The Postfix Configuration Files

[POST=236399]4.4 Syslog Configuration[/POST]
4.5 Disable Sendmail and Start-Up Dovecot/Postfix

[POST=236401]5. Advanced Features[/POST]

5.1 Web Mail

5.1.1 Installations for Roundcube Web Mail
5.1.2 Configuration of Web Mail with Roundcube
[POST=236402]5.1.3 Roundcube with HTTP Digest Authentication[/POST]
[POST=236404]5.1.4 The Roundcube HTTP Digest Authentication Plugin[/POST]

The Topics below are not treated yet:
5.2 Blocking Spam
5.2.1 Before queue validation
5.2.2 Enabling Grey Listing
5.2.3 Statistical Spam Filter using CRM114 (eventually before queue)
5.3 Try encrypted direct delivery first, and use the SMTP relay only as fallback

6. Wish list for the Future
6.1 Digest-SHA3 authentication scheme for Web and Mail
6.2 Autonomous X.509 PKI
6.3 Direct End-to-end delivery + TLS
6.4 ...

1. Introduction

Many people believe that Plain authentication is secure once used together with TLS, and therefore they leave out the hassle of setting up an inherently more secure authentication mechanism.

The Dovecot documentation wiki discusses more security considerations and in this respect it is specially important to understand the relationship between the Authentication Mechanism and the Password Storage scheme, one of which must be for technical reasons Plain Text (or near to Plain Text in respect to the security quality).

So we may either have encrypted authentication with plain text (or weakly hashed) passwords in the password store, or we may have plain text authentication with strongly salted+hashed passwords in the password store on the server. The Dovecot conclusion seems to be to go with the latter, i.e. to have plain text authentication, however secured by TLS, and to stay with the benefit of a highly secured password store.

This is not necessarily a bad choice, specially if the server admin sees a certain danger that the password store can be accessed by an unauthorized 3rd party. In this case, it would be really good to have the passwords salted+hashed as strongly as possible, and as god will, let TLS secure the plain text authentication. Most probably this is the choice for small to medium companies where a considerable number of persons have access to the mail server, and who didn't set up a separate e.g. Kerberos password server. There are other points of views though:

First, I am talking about a home mail server, and if 3rd parties ever would get access to it and come even close to the password store, then the damage would be already that huge that access to the passwords would be one of my smallest concerns.

Second, even in a small company, I would setup separate servers for mail, file/web/others, and passwords.

Third, there are advanced technics of TLS interception. Companies tend to install DPI firewalls, showing a valid certificate to internal clients, so the firewall can decrypt the packets, filter on it, and encrypt it again for the transport to the final destination. So, accessing your home mail server from behind a DPI firewall potentially shows your password in clear to the company's admin(s).

Fourth, in principal said kind of DPI would work also on a larger scale, a potential MITM for example sitting in Cheltenham would only need to offer a somehow valid certificate to the client and could decrypt the traffic. Another consideration is that most TLS connections are still not providing Perfect Forward Secrecy (s. also SSL: Intercepted today, decrypted tomorrow). That means, government attackers may intercept the message, and obtain the private keys from the CA by the way of, for example a NSL accompanied with a gag order. With the private key, the whole traffic could be decrypted later on. In both cases any clear text passwords are revealed, and could be used for secret logins in order to get access to all the mails stored at that IMAP mail server for that user.

These considerations let me not to follow the suggestion of the Dovecot documentation. I choose to setup my mail server with the CRAM-MD5 authentication mechanism, staying with the drawback that the passwords in the store may only be weakly hashed by MD5.

I choose CRAM-MD5 because all of my clients (Mac OS X 10.8 and iOS 7) do support this for IMAP, POP3, and SMTP-Auth. Even, if MD5 is nowadays vulnerable to brute force attacks by its own, in the CRAM-MD5 scheme, it can be still considered safe against online attacks, i.e. cracking the scheme in the course of the connection. However, an offline dictionary or brute force attack to recover the password seems to be feasible after intercepting a successful CRAM-MD5 protocol exchange.

Yes, CRAM-MD5 and Digest-MD5 came to age, and I would love to see a Digest-SHA512 authentication mechanism, but it doesn't exit yet and we have to live with what we have. It is worth to note, that MD5 is not completely useless. The occasional brute force cracker would already have a hard time for passwords with more than 10 characters in length, and also super computers would take a reasonable amount of dedicated time for brute-forcing a 13 char password. Reasonable means, that it didn't finish until you rotated your password anyway, let's say once in a month. Dedicated means, that this computer wouldn't do anything else than trying to crack your password. The budget of the best equipped Secret Service would be rapidly exhausted, if they want to do this for millions of intercepted CRAM-MD5 sessions.

In addition to length (>11 chars), the usual advises for password strength keep-on being valid - use the full character set, i.e. alphanumerics, punctation, accented chars, etc. In this respect it helps a lot to switch from short Passwords to long Passphrases. Something like:

"â‰¤Ach wie gut, daÃŸ niemand weiÃŸ, daÃŸ ich Rumpelstilzchen heiÃŸ.â‰¥"

Again, it is advised to change this frequently.

Anyway, I forced my server to use TLS ciphers that provide Perfect Forward Secrecy. For this, I need to install OpenSSL from the ports. In addition, keeping NSLs accompanied with gag orders in mind, it is nowadays a better choice to only rely on your own certification authority, i.e., self sign your own certificates, and don't trust anybody.

Anonymous · Oct 13, 2013

2. Requirements

2.1 The ISP must not block port 25

You need a FreeBSD server connected to the internet, and your ISP must not block the incoming SMTP port 25. A fixed IP address is not needed, but doesn't harm either.

You don't want to go into the big effort of setting up your home mail server, only to find out that it is useless because port 25 is blocked, so it is better to check it before. Connect a *nix machine to the internet by the way of the public endpoint that you want to check. If you connect it by the way of a router, then make sure that port 25 is NAT-redirected to said machine. Also you need allow traffic on port 25 on your firewall(s) for that test machine.

On the command line of the test box execute as user root # nc -lk 25.
Enter the web browser on the machine, and point it to: http://www.canyouseeme.org.
Verify that the web page shows the correct public IP.
Enter 25 into the text field and press the button <Check Your Port>.

The following message indicates that you can go ahead with setting up your Home Mail Server:

Success: I can see your service on 123.456.78.90 on port (25), Your ISP is not blocking port 25

The following message indicates a need for trouble shooting, and perhaps for asking your ISP or another ISP for a different contract.

Error: Error: I could not see your service on 123.456.78.90 on port (25) Reason: Connection timed out

2.2 The Domain Name

You need your own domain hosted by a Hosting Provider having the following characteristics:

dynamic DNS configuration for that domain including MX records
basic mail service, so the provider would accept authenticated SMTP relaying (optionally, as fallback only) of your outgoing messages (you won't need anything else of the mail service)

2.3 Dynamic DNS Configuration

The Postfix SMTP server must be able to receive e-mail messages from the outside on the SMTP port 25, and the Postfix smtpd must listen on the public interface of our server on port 25 and may in addition listen also on the submission port 587.

In any case, outside SMTP servers must know, that your host is accepting mail for your domain, and this shall be announced by your DNS A and MX records (Addresse and Mail EXchanger) of your domain.

Usually, home installations don't have fixed public IP addresses, but IP addresses that are dynamically assigned by the ISP, and that are subject to change at any time. The DNS records A and MX of the domain must follow these assignments, otherwise chances are that foreign mail servers try to send messages for your domain to wrong IP addresses.

The DynDNS update API has been adopted by many domain & mail hosters. DynDNS update is initiated by a simple HTTP(S) GET request. Having the dynamic DNS URL and authentication credentials from your domain hosting provider, it is easy to check whether DNS updating works using a standard web-browser.

2.3.1 Checking Dynamic DNS

Let's assume, that the second level domain is example.com and its sub-domain mail.example.com has been setup at a domain & mail hosting provider like Strato in Berlin. For executing a one-time DNS update, you could enter in the address bar of the web browser the following:

Code:

https://[color="blue"]dyndns.strato.com[/color]/nic/update?hostname=[color="blue"]example.com[/color],mail.[color="blue"]example.com[/color]&myip=[color="blue"]123.456.78.90[/color]&mx=mail.[color="blue"]example.com[/color]

You need to replace the URL of Strato by the URL of your domain provider, example.com by your actual domain, and 123.456.78.90 by your actual public IP. After entering your credentials the following simple message should appear in the browser window:

Code:

good 123.456.78.90

Then at the command line of your terminal execute $ host example.com; host mail.example.com, again replacing example.com by your second level domain.

Code:

example.com has address 123.456.78.90
example.com mail is handled by 10 mail.example.com.
mail.example.com has address 123.456.78.90
mail.example.com mail is handled by 10 mail.example.com.

Please run the above tests! If the dynamic DNS update doesn't work by the way of a web browser, then everything else below won't work either.

2.3.2 Creating a Dynamic DNS Request File and the Update Script

Install converters/base64. We need it in order to create the base 64 hash of the credentials for accessing the dynamic dns update server. Replace dyndns_login:dynsdns_password by your actual login

assword credentials.
# cd /usr/ports/converters/base64
# make install clean
# echo -n "dyndns_login:dynsdns_password" | base64 -e

Code:

ZHluZG5zX2xvZ2luOmR5bnNkbnNfcGFzc3dvcmQ=

Create the directory /usr/local/etc/dyndns.d
# mkdir /usr/local/etc/dyndns.d
# cd /usr/local/etc/dyndns.d

Create and edit the file /usr/local/etc/dyndns.d/example.com.req and enter the following content:

Code:

GET /nic/update?hostname=[color="blue"]example.com[/color],mail.[color="blue"]example.com[/color]&myip=NEWIP&mx=mail.[color="blue"]example.com[/color] HTTP/1.0
Host: [color="blue"]dyndns.strato.com[/color]
Authorization: Basic [color="blue"]ZHluZG5zX2xvZ2luOmR5bnNkbnNfcGFzc3dvcmQ=[/color]
User-Agent: [color="blue"]example.com[/color] - Mozilla/5.0 (alike) - 11

Replace the blue settings by your actual domain, dyndns host, and credential hash.

Create and edit the file /usr/local/etc/dyndns.d/update.sh, adding the following content and again by replacing the blue settings by your actual ones.

Code:

#!/bin/sh

if [ -e /var/tmp/currentIP ] ; then
    currentIP=`/bin/cat /var/tmp/currentIP`
else
    currentIP=
fi

if [ "$1" = "" ] ; then
    newIP=`/sbin/ifconfig [color="blue"]em1[/color] | /usr/bin/sed -n '/.inet /{s///;s/ .*//;p;}'`
else
    newIP=$1
fi

if [ "$newIP" = "" -o "$newIP" = "$currentIP" ] ; then
    echo "nochg $currentIP"

else
    /usr/bin/sed "s/NEWIP/$newIP/" /usr/local/etc/dyndns.d/[color="blue"]example.com[/color].req \
    | /usr/bin/openssl s_client -connect [color="blue"]dyndns.strato.com[/color]:443 -tls1_2 -cipher HIGH@STRENGTH -CAfile /usr/local/share/certs/ca-root-nss.crt -quiet -crlf

    echo "$newIP" > /var/tmp/currentIP
fi

# chmod ugo+x update.sh

If /usr/local/share/certs/ca-root-nss.crt does not exist on your machine, then install the port security/ca_root_nss.
# cd /usr/ports/security/ca_root_nss
# make install clean

Test the dyndns update script, replacing the parameter 123.456.78.90 by your actual public IP:
# /usr/local/etc/dyndns.d/update.sh 123.456.78.90

Code:

depth=3 C = ZA, ST = Western Cape, L = Cape Town, O = Thawte Consulting cc, OU = Certification Services Division, CN = Thawte Premium Server CA, emailAddress = premium-server@thawte.com
verify return:1
depth=2 C = US, O = "thawte, Inc.", OU = Certification Services Division, OU = "(c) 2006 thawte, Inc. - For authorized use only", CN = thawte Primary Root CA
verify return:1
depth=1 C = US, O = "Thawte, Inc.", CN = Thawte SSL CA
verify return:1
depth=0 C = DE, ST = Berlin, L = Berlin, O = Strato AG, OU = Rechenzentrum, CN = *.strato.com
verify return:1
HTTP/1.0 200 OK
Date: Tue, 22 Oct 2013 21:51:01 GMT
Server: Apache/1.3.41 (Unix) mod_deflate/1.0.21 mod_perl/1.31 mod_ssl/2.8.31 OpenSSL/0.9.8o
Connection: close
Content-Type: text/plain; charset=ISO-8859-1

good 123.456.78.90
good 123.456.78.90
read:errno=0

2.3.3 Utilizing the Update Script for Automatic Dynamic DNS Updating

You need to feed the update script that was created in the foregoing chapter with one parameter, that is the public IP address of the FreeBSD machine. Depending on how it is connected to the internet the public IP must be obtained by different methods.

My machine is connected directly to the cable modem and gets its public IP via DHCP. In the course of renewing its lease, the dhclient daemon calls the script /etc/dhclient-exit-hooks with the current public IP placed into the variable $new_ip_address, see dhclient-script(8)(). In this case the preferred method is to call /usr/local/etc/dyndns.d/update.sh feeding $new_ip_address directly from /etc/dhclient-exit-hooks. So, the update happens immediately if $new_ip_address is different form the one stored at /var/tmp/currentIP.

/etc/dhclient-exit-hooks looks like:

Code:

#!/bin/sh

case "$new_ip_address" in
10.*) ;;
172.1[6-9].* | 172.2[0-9].* | 172.3[0-1].*) ;;
192.168.*) ;;
"") ;;
*)
    /usr/bin/logger -t dyndns verify $new_ip_address
    /usr/local/etc/dyndns.d/update.sh $new_ip_address > /dev/null 2>&1
    ;;
esac

Many ways lead to Rome.

If your machine is connected by the way of PPP(oE) then you can let /usr/local/etc/dyndns.d/update.sh obtain the current IP from the respective interface via calling ifconfig. In this case replace in the script em1 by the ppp interface, presumably ppp0, and call /usr/local/etc/dyndns.d/update.sh without passing an IP address parameter once a connection has been established.

If your machine is connected by other means, then you have to find out a way of reliably obtaining the public IP and of being notified once it changed for then passing it to /usr/local/etc/dyndns.d/update.sh either on the command line or in a cron job.

Anonymous · Oct 13, 2013

3. Installations

3.1 Installation of OpenSSL from the ports, and re-building of all ports depending on OpenSSL

OpenSSL from the base system does not offer the cipher suites that provide Perfect Forward Secracy. Therefore, we want to install OpenSSL from the ports and then we need to update all ports that depend on it.

# cd /usr/ports/security/openssl
# make install clean

I configured OpenSSL as follows:

Code:

+----------------------------- openssl-1.0.1_8 --------------------------------+                                         
|+---------------------------------------------------------------------------+ |  
| | [x] SHARED   build of shared libs                                        | |
| | [x] THREADS  Threading support                                           | |
| | [ ] I386     Optimize for i386 (instead of i486+)                        | |  
| | [x] SSE2     runtime SSE2 detection                                      | |  
| | [x] ASM      optimized Assembler code                                    | |  
| | [ ] PADLOCK  VIA Padlock support                                         | |  
| | [x] ZLIB     zlib compression support                                    | |  
| | [x] SCTP     SCTP protocol support                                       | |  
| | [ ] MD2      MD2 hash (obsolete)                                         | |  
| | [x] RC5      RC5 cipher (patented)                                       | |  
| | [x] RFC3779  RFC3779 support                                             | |  
| | [ ] GMP      gmp support (LGPLv3)                                        | |  
| | [x] EC       Optimize NIST elliptic curves                               | |  
| +--------------------------------------------------------------------------+ |  
+------------------------------------------------------------------------------+

Now all ports that use OpenSSL libraries from the base system, either libssl.so.6 or libcrypto.so.6 need to be rebuild. Here comes a semi-automatic way to complete this task by using portmaster(8)(). The first step is to obtain a list of the affected ports, by executing the following shell pipeline. Be prepared that this may take several minutes for completion.

 # find /usr/local/bin /usr/local/sbin /usr/local/libexec /usr/local/lib -type f | xargs -n1 file -F ' ' | grep ELF | cut -f1 -d' ' |  xargs ldd -f '%A %o\n' | grep "libssl.so.6\|libcrypto.so.6" | cut -f1 -d' ' | sort -u | xargs -n1 pkg_info -W | cut -f6 -d' ' | sort -u | tee ~/openssl_dependencies.txt

The dependency list is output on the terminal and because of the final tee command the list is also put into the file ~/openssl_dependencies.txt that after revision can be feed into portmaster(8)(). Here comes the output on my system. The list on your system might vary depending on the ports that are installed.

Code:

apache24-2.4.6
apr-1.4.8.1.5.2
curl-7.32.0
libevent2-2.0.21
netatalk-2.2.5,1
openldap-client-2.4.36
[color="orange"]openssl-1.0.1_8[/color]
php5-curl-5.4.20_1
php5-openssl-5.4.20
py27-ldap2-2.4.13
python27-2.7.5_3
serf-1.2.1_1
squid-3.3.9
subversion-1.8.3

The second step is to revise the list ~/openssl_dependencies.txt, for example I removed the openssl entry. The third step is to issue the following command for letting portmaster rebuild all ports depending on OpenSSL:

# portmaster `cat ~/openssl_dependencies.txt`
# rm ~/openssl_dependencies.txt

3.2 Installation of Dovecot IMAP Server + Dovecot SASL

# cd /usr/ports/mail/dovecot2
# make install clean

Select the following configuration options:

Code:

+------------------------------ dovecot-2.2.6 ---------------------------------+  
| +--------------------------------------------------------------------------+ |  
| | [x] DOCS      Build and/or install documentation                         | |  
| | [x] EXAMPLES  Build and/or install examples                              | |  
| | [ ] GSSAPI    GSSAPI support                                             | |  
| | [x] KQUEUE    kqueue(2) support                                          | |  
| | [ ] LDAP      LDAP support                                               | |  
| | [ ] LIBWRAP   TCP wrapper support                                        | |  
| |+[ ] LUCENE    CLucene FTS support                                        | |  
| | [ ] MYSQL     MySQL database support                                     | |  
| | [ ] PGSQL     PostgreSQL database support                                | |  
| | [ ] SOLR      Solr FTS support                                           | |  
| | [ ] SQLITE    SQLite database support                                    | |  
| | [x] SSL       SSL protocol support                                       | |  
| | [ ] VPOPMAIL  vpopmail support                                           | |  
| +--------------------------------------------------------------------------+ |  
+------------------------------------------------------------------------------+

3.3 Installation of Cyrus SASL2

Although, Dovecot comes with its own SASL library, Cyrus SASL2 needs to be installed because Postfix needs it for SMTP authentication for passing outgoing mail to the SMTP (relay) server of the domain/mail hosting provider.

# cd /usr/ports/security/cyrus-sasl2
# make install clean

Select the following configuration options:

Code:

+--------------------------- cyrus-sasl-2.1.26_2 ------------------------------+
| +--------------------------------------------------------------------------+ |
| | [ ] ALWAYSTRUE          the alwaystrue password verifier                 | |
| | [ ] AUTHDAEMOND         use of authdaemon                                | |
| | [ ] KEEP_DB_OPEN        Keep handle to Berkeley DB open                  | |
| | [ ] OBSOLETE_CRAM_ATTR  cmusaslsecretCRAM-MD5 property                   | |
| | [ ] BDB                 Berkeley DB support                              | |
| | [ ] MYSQL               MySQL database support                           | |
| | [ ] PGSQL               PostgreSQL database support                      | |
| |----------------------------------- MECH ---------------------------------| |
| | [x] CRAM                CRAM-MD5 authentication                          | |
| | [x] DIGEST              DIGEST-MD5 authentication                        | |
| | [x] LOGIN               LOGIN authentication                             | |
| | [x] NTLM                NTLM authentication                              | |
| | [x] OTP                 OTP authentication                               | |
| | [x] PLAIN               PLAIN authentication                             | |
| | [x] SCRAM               SCRAM authentication                             | |
| |------------------------- SQLite database support ------------------------| |
| | ( ) SQLITE2             SQLite 2 database                                | |
| | ( ) SQLITE3             SQLite 3 database support                        | |
| +--------------------------------------------------------------------------+ |
+------------------------------------------------------------------------------+

3.4 Postfix Mail Server

cd /usr/ports/mail/postfix
make install clean

Select the following configuration options:

Code:

+----------------------------- postfix-2.10.1,1 -------------------------------+
| +--------------------------------------------------------------------------+ | 
| | [x] BDB        Berkeley DB (uses WITH_BDB_VER)                           | | 
| | [ ] CDB        CDB maps lookups                                          | | 
| | [ ] INST_BASE  Install into /usr and /etc/postfix                        | | 
| | [ ] LDAP_SASL  OpenLDAP client-to-server SASL auth                       | | 
| | [ ] MYSQL      MySQL maps (uses WITH_MYSQL_VER)                          | | 
| | [ ] NIS        NIS maps lookups                                          | | 
| | [ ] OPENLDAP   OpenLDAP maps (uses WITH_OPENLDAP_VER)                    | | 
| | [x] PCRE       Perl Compatible Regular Expressions                       | | 
| | [ ] PGSQL      PostgreSQL maps (uses DEFAULT_PGSQL_VER)                  | | 
| | [x] SASL2      Cyrus SASLv2 (Simple Auth. and Sec. Layer)                | | 
| | [x] SPF        SPF support (via libspf2 1.2.x)                           | | 
| | [ ] SQLITE     SQLite maps                                               | | 
| | [ ] TEST       SMTP/LMTP test server and generator                       | | 
| | [x] TLS        SSL and TLS support                                       | | 
| | [ ] VDA        VDA (Virtual Delivery Agent 32Bit)                        | | 
| |------------------- Dovecot SASL authentication methods ------------------| | 
| | ( ) DOVECOT    Dovecot 1.x SASL authentication method                    | | 
| | (*) DOVECOT2   Dovecot 2.x SASL authentication method                    | | 
| |-------------- Kerberos network authentication protocol type -------------| | 
| | ( ) SASLKRB5   If your SASL req. Kerberos5, select this                  | | 
| | ( ) SASLKMIT   If your SASL req. MIT Kerberos5, select this              | | 
| +--------------------------------------------------------------------------+ | 
+------------------------------------------------------------------------------+

If not already installed, then pcre, the Perl Compatible Regular Expressions library will be build and installed in the course of the Postfix installation. In the configuration dialog of pcre, disable STACK_RECURSION, since this may cause stack overflows, and it is a good idea not to run that risk in an un-attended MTA processing input from unknown sources.

Code:

+-------------------------------- pcre-8.33 -----------------------------------+
| +--------------------------------------------------------------------------+ |
| |+[ ] STACK_RECURSION  Use the stack for recursion during matching         | |
| +--------------------------------------------------------------------------+ |
+------------------------------------------------------------------------------+

Code:

...
Would you like to activate Postfix in /etc/mail/mailer.conf [n]? [color="blue"]y[/color]
...

Answer the final question with y(es), and the file /etc/mail/mailer.conf will adapt your system to Postfix instead of the default Sendmail.

Code:

#
# Execute the Postfix sendmail program, named /usr/local/sbin/sendmail
#
sendmail        /usr/local/sbin/sendmail
send-mail       /usr/local/sbin/sendmail
mailq           /usr/local/sbin/sendmail
newaliases      /usr/local/sbin/sendmail

Anonymous · Oct 13, 2013

4. Configurations

Dovecot and Postfix offer a huge variety of configuration options which may serve for any imaginable mail server constellation. We are going to set up a simple Home Mail Server and want to stay with the default settings as far as possible. Anyway, some decision must be taken:

All mail services shall be secured by TLS using self-signed certificates, and the store shall be at /usr/local/etc/certs/.
The mail services shall be offered to virtual users that are different from the system users.
The expected small number of users does not justify a special backend for the user database (like LDAP or xSQL), so we stay with a simple passwd-style file.
For the mail store we create the new directory tree at /var/mail/users/.

For the following we assume, that the mail service that we are going to set up is for the domain example.com. The MX is mail.example.com. Both domains resolve by the way of dynamic DNS updates to the public IP of our FreeBSD machine.

4.1 Creating a self-signed certificate chain

If you followed the third post of this thread, then you got 2 OpenSSL installations on your system, and in order to make sure that you are using the tools of the new one, without needing to type the full prefix all the time, I recommend to create the following symbolic link:

# cd /usr/local/bin; ln -s openssl ossl; cd

Then check it: # ossl version

Code:

OpenSSL 1.0.1e 11 Feb 2013

Nowadays some web browsers show a warning on SSL certificates genereated with 1024 bits or less, so we want the key and certificate for our CA (Certification Authority) being generated with 2048 bits. Either we need to create the whole CA hierarchy manually or we need to edit /usr/local/openssl/openssl.cnf and change the setting default_bits to 2048.
# nano /usr/local/openssl/openssl.cnf

Code:

...
[ req ]
default_bits            = 2048
...

Step 1 Create the directory for your certificate store.
# mkdir /usr/local/etc/certs
# cd /usr/local/etc/certs

Step 2 Let the CA.pl Perl script create the CA hierarchy, which consist of its private key, the public CA certificate and some house holding files. The blue text represent user entries.
# OPENSSL=/usr/local/bin/openssl /usr/local/openssl/misc/CA.pl -newca

Code:

CA certificate filename (or enter to create)

Making CA certificate ...
Generating a 2048 bit RSA private key
...................+++
..................................................................................................................................................................................+++
writing new private key to './demoCA/private/cakey.pem'
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:[color="blue"]DE[/color]
State or Province Name (full name) [Some-State]:[color="blue"]Bayern[/color]
Locality Name (eg, city) []:[color="blue"]Regensburg[/color]
Organization Name (eg, company) [Internet Widgits Pty Ltd]:[color="blue"]Familie Mustermann[/color]
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:[color="blue"]example.com[/color]
Email Address []:[color="blue"]hostmaster@example.com[/color]

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
Using configuration from /usr/local/openssl/openssl.cnf
Enter pass phrase for ./demoCA/private/cakey.pem:
Check that the request matches the signature
Signature ok
Certificate Details:
        Serial Number: 11204118603972774038 (0x9b7d068223248896)
        Validity
            Not Before: Oct 16 23:02:58 2013 GMT
            Not After : Oct 15 23:02:58 2016 GMT
        Subject:
            countryName               = DE
            stateOrProvinceName       = Bayern
            organizationName          = Familie Mustermann
            commonName                = example.com
            emailAddress              = hostmaster@example.com
        X509v3 extensions:
            X509v3 Subject Key Identifier: 
                00:48:A9:A7:BE:13:CE:50:9D:14:51:72:8F:57:55:2A:49:49:47:0A
            X509v3 Authority Key Identifier: 
                keyid:00:48:A9:A7:BE:13:CE:50:9D:14:51:72:8F:57:55:2A:49:49:47:0A

            X509v3 Basic Constraints: 
                CA:TRUE
Certificate is to be certified until Oct 15 23:02:58 2016 GMT (1095 days)

Write out database with 1 new entries
Data Base Updated

Step 3 Create and sign the certificates for the mail services.
Generate the private key, which in this case must not be encrypted, i.e. it must be accessible without a password.
# ossl genrsa -out mailservice.key 2048

Code:

Generating RSA private key, 2048 bit long modulus
....................................................................................................................................+++
............................+++
e is 65537 (0x10001)

Create the certificate signing request.
# ossl req -new -key mailservice.key -out mailservice.csr

Code:

You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:[color="blue"]DE[/color]
State or Province Name (full name) [Some-State]:[color="blue"]Bayern[/color]
Locality Name (eg, city) []:[color="blue"]Regensburg[/color]
Organization Name (eg, company) [Internet Widgits Pty Ltd]:[color="blue"]Familie Mustermann[/color]
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:[color="blue"]mail.example.com[/color]
Email Address []:[color="blue"]hostmaster@example.com[/color]

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

Step 4 Ask your CA to sign the certificate.
# ossl ca -days 1094 -in mailservice.csr -out mailservice.crt

Code:

Using configuration from /usr/local/openssl/openssl.cnf
Enter pass phrase for ./demoCA/private/cakey.pem:
Check that the request matches the signature
Signature ok
Certificate Details:
        Serial Number: 11204118603972774039 (0x9b7d068223248897)
        Validity
            Not Before: Oct 16 23:26:13 2013 GMT
            Not After : Oct 14 23:26:13 2016 GMT
        Subject:
            countryName               = DE
            stateOrProvinceName       = Bayern
            organizationName          = Familie Mustermann
            commonName                = mail.example.com
            emailAddress              = hostmaster@example.com
        X509v3 extensions:
            X509v3 Basic Constraints: 
                CA:FALSE
            Netscape Comment: 
                OpenSSL Generated Certificate
            X509v3 Subject Key Identifier: 
                79:C9:75:C6:70:9C:21:B3:03:0F:90:D0:8A:11:50:B2:F8:51:64:D2
            X509v3 Authority Key Identifier: 
                keyid:00:48:A9:A7:BE:13:CE:50:9D:14:51:72:8F:57:55:2A:49:49:47:0A

Certificate is to be certified until Oct 14 23:26:13 2016 GMT (1094 days)
Sign the certificate? [y/n]:[color="blue"]y[/color]


1 out of 1 certificate requests certified, commit? [y/n][color="blue"]y[/color]
Write out database with 1 new entries
Data Base Updated

Step 5 Put a copy of your CA certificate at the root level of the certificate store, and join it to the service certificate in order to create the service certificate chain.
# cp demoCA/cacert.pem ca.crt
# cat mailservice.crt ca.crt > mailservice.chn

Step 6 Generate your own set of Diffie-Hellman parameters in order to harden the DH ephemeral key exchange which provides for Perfect Forward Secrecy.
# ossl gendh -out dh_512.pem -2 512
# ossl gendh -out dh_1024.pem -2 1024

Step 7 Finally, strip off all access rights from the certificate store.
# chmod -R 000 /usr/local/etc/certs
# ls -l /usr/local/etc/certs

Code:

total 20
----------  1 root  wheel  4496 Oct 16 20:02 ca.crt
d---------  6 root  wheel   512 Oct 16 20:00 demoCA
----------  1 root  wheel   245 Oct 16 20:37 dh_1024.pem
----------  1 root  wheel   156 Oct 16 20:37 dh_512.pem
----------  1 root  wheel  9140 Oct 16 20:30 mailservice.chn
----------  1 root  wheel  4641 Oct 16 20:26 mailservice.crt
----------  1 root  wheel  1062 Oct 16 20:19 mailservice.csr
----------  1 root  wheel  1679 Oct 16 20:17 mailservice.key

Your self-signed certificates are in place now, waiting for being used by Dovecot and Postfix.

4.2 Configuration of Dovecot IMAP/POP3

4.2.1 The Virtual Users Store

The mail users in this setup are virtual users, i.e. they are not linked to system users. Mail users are registered in a separate Virtual User Store, which may be a simple file or a LDAP or xSQL database. The dovecot/imap daemon accesses on behalf of the virtual user his mails, and the virtual user would have no access by other means.

Dovecot authenticates users against the user names and passwords in the Virtual User Store. A user entry may contain a uid and gid value, and if present, dovecot/imap accesses the users mails with that uid/gid pair. It is recommended to assign to each virtual user a unique uid. This one, let's call it v-uid, would be valid for the file system, i.e. dovecot/imap may use it for read/write access to the virtual users mails, but this v-uid would ideally have no correspondent among the system users.

I recommend to choose 10000 as the first valid v-uid and v-gid. In order to manifest that choice, let us create one system user and one system group having these id's. Technically, this won't have any effect, but it has the benefit that the Virtual Mail Users got a mark in the regular user database, which later on may make the admin remember that virtual users do exist on the system.

# pw groupadd virtmail -g 10000
# pw useradd virtuser -u 10000 -g 10000 -c "Virtual Mail User" -d /var/empty -s /usr/sbin/nologin
# pw groupmod virtmail -m virtuser

The login name of a mail user is his/her e-mail address, e.g. rolf@example.com. For the CRAM-MD5 authentication scheme, the hash of the password is stored, and the hash can be obtained using doveadm pw. For example, with the incredible password "rolfheinrich": # doveadm pw -p rolfheinrich

Code:

{CRAM-MD5}cf8527ae5867fc08068952261363c49f0bcc0d1d5f166bbd20d3783b3e97f828

For adding a new user to the mail users file, do the following:
# cd /usr/local/etc/dovecot
# printf "rolf@example.com:%s:10001:10000:\n" `doveadm pw -p "rolfheinrich"` >> users

If the file /usr/local/etc/dovecot/users doesn't yet exist, it would be created by this command, otherwise the new user would be simply added on a new line at the end of the file.

Code:

rolf@example.com:{CRAM-MD5}cf8527ae5867fc08068952261363c49f0bcc0d1d5f166bbd20d3783b3e97f828:10001:10000:
...

The format of the virtual users file is:

Code:

<login name>:<password hash>:<v-uid>:<v-gid>:<home directory>

Assign a unique v-uid starting at 10001 to each user, and stay with the base v-gid of 10000 for everyone. Leave the field for the home directory empty, i.e. the trailing colon.

Anonymous · Oct 13, 2013

4.2 Configuration of Dovecot IMAP/POP3 (continued)

4.2.2 The Mail Directory

In /var/mail create the new directory users:
# mkdir /var/mail/users

By standard, mails that are send to system users, mainly root are stored in plain mbox files at the root level of /var/mail - one file for each user, and its file name is the users name. The new sub-directory users will receive and store the mails for the virtual mail accounts.

Dovecot can store mails individually in directories or in mbox files. A mbox file contains all the mails appended one to another in the sequence of entrance. This format is adequate for mail stores that are not likely to change heavily. However, a IMAP server allows its users to organize the mails in sub-directories. The user may keep some mails, delete others, mark mails as junk that are deleted later, store drafts that are moved to the sent box later, etc. Therefore, it is better, to have the mails stored individually in respective sub-directories. Thereby, for example, moving mails around is basically a file system operation, which is much more effective compared to extracting a chunk from the middle of a huge text file, and insert this chunk somewhere into another huge text file. Our mails wil be stored in the Maildir format.

For now, adjust the correct access rights of the new directory /var/mail/users.
# chown virtuser:virtmail /var/mail/users
# chmod 770 /var/mail/users

Dovecot automatically populates this mail directory with the mailboxes of the virtual users when appropriate.

4.2.3 The Dovecot Configuration File

In the previous chapters all pre-requisites were set up, and so Dovecot itself can be configured quickly. Create and edit the file /usr/local/etc/dovecot/dovecot.conf.
# nano /usr/local/etc/dovecot/dovecot.conf

Code:

auth_realms        = [color="blue"]example.com[/color]
auth_default_realm = [color="blue"]example.com[/color]
auth_mechanisms    = cram-md5

first_valid_uid = [color="blue"]10000[/color]
first_valid_gid = [color="blue"]10000[/color]

mail_location = maildir:/var/mail/users/%u

userdb {
  args = username_format=%u /usr/local/etc/dovecot/users
  driver = passwd-file
}

passdb {
  args = username_format=%u /usr/local/etc/dovecot/users
  driver = passwd-file
}

namespace inbox {
  inbox = yes
  location = 
  mailbox Drafts {
    special_use = \Drafts
  }
  mailbox Junk {
    special_use = \Junk
  }
  mailbox Sent {
    special_use = \Sent
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
  mailbox Trash {
    special_use = \Trash
  }
  mailbox "Deleted Messages" {
    special_use = \Trash
  }
  prefix = 
}

service auth {
  unix_listener /var/spool/postfix/private/auth {
    user  = postfix
    group = postfix
    mode  = 0666
  }
}

service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    user  = postfix
    group = postfix
    mode  = 0660
  }
}

ssl_cert        = </usr/local/etc/certs/mailservice.chn
ssl_key         = </usr/local/etc/certs/mailservice.key
ssl_cipher_list = TLSv1+HIGH:!3DES:!CAMELLIA:!aNULL:@STRENGTH

syslog_facility = [color="blue"]local1[/color]

Replace example.com by your actual domain.
The only allowed authentication mechanism is cram-md5.
Dovecot doesn't serve system users as indicated by the first_valid_xid settings.
The mail store is /var/mail/users, and each user owns a mailbox %u.
The virtual mail users are registered in the passwd-like file /usr/local/etc/dovecot/users
Each mailbox contains some special sub-directories, whose names shall match the respective names used by the client.
Dovecot provides user authentication and local transport for Postfix.
The self-signed certificate chain is assigned, and strong SSL ciphers are preferred.
Use syslog for logging. Choose a free facility. syslog will be configured in a later chapter.

4.3 Configuration of the Postfix Mail Transfer Agent

4.3.1 Relay Host Configuration

The Postfix MTA is supposed to receive mails directly from the internet. For this, the first entry in the MX record of your domain shall point to the public IP address of the Postfix host (s. chapter 2.3 Dynamic DNS Configuration). In addition, your ISP and your Firewall must not block the incoming port 25. However, many peers would not accept outgoing mails directly from your mail server, as long you don't run it on a fixed IP address with an associated valid PTR record allowing reverse resolution of your public IP to your domain. Therefore, we setup Postfix to pass outgoing mail to a relay host, which then distributes it to the final recipients. Usually you would contract a simple mail service together with the domain hosting service with your hosting provider.

Your hosting provider informed to you a domain name of its outgoing SMTP server, e.g. smtp.maildomainhoster.net, and you received SMTP-Auth login credentials for passing your outgoing mails to this server. Postfix acts in this respect pretty much like a normal e-mail client, when sending out mail. Your hosting provider won't even see a difference.

Create and edit the file /usr/local/etc/postfix/relay-sasl-password entering the your login credentials:
# cd /usr/local/etc/postfix
# nano relay-sasl-password

Code:

[[color="blue"]smtp.maildomainhoster.net[/color]][FILE]:submission[/FILE] [color="blue"]admin_user@example.com[/color]:[color="blue"]plain-password[/color]

Replace the items in blue with the real SMTP host name and with your exact login credentials. You would indicate a admin user who is allowed to send e-mails on behalf of any user of your domain. The above assumes that the relay host accepts mail on the submission port 587. If this is not the case, then remove :submission after the closing square bracket. The file must be mapped for Postfix can use it.
# postmap relay-sasl-password

Note: relay-sasl-password must be re-mapped each time it has been edited.

4.3.2 Local Recipients and Mail Aliases

We want Postfix to accept incoming mail for local recipients only, and therefore we need to tell postfix somehow who the local recipients are. For this we can simply map the users file that has been created in the course of setting up the virtual mail users. Since the file contains information that is not need for said purpose, we pass it through sed(1)() which strips off the password hashes, etc., before mapping.
# cd /usr/local/etc/dovecot
# sed 's/:.*/ exists/' users > tmp; postmap tmp; mv tmp.db users.db; rm tmp

Note: The file users must be re-mapped for Postfix each time it has been changed.

Postfix considers the users listed in the system file /etc/aliases also as local recipients. The default /etc/aliases tells the system to redirect all system messages to the user root. For your convenience add to aliases one line, which redirects system mails to root to a virtual mail user, and thereby, the results of the nightly security check and of the other periodic scripts would be passed to a normal mail user. Depending on your preferences, you would add a separate admin user in the same way you add any other user to the virtual mail users file, and redirect mails to root to that virtual admin user, or you simply redirect it to your own mail account. I did the latter, and I added to /etc/aliases at the very end the following line:

Code:

root:           rolf

This file must also be mapped for Postfix, and in this case we use the sendmail compatible command # newaliases

4.3.3 The Postfix Configuration Files

All pre-requisites are discussed in the previous chapters, and finally Postfix can be setup by editing its two configuration files /usr/local/etc/postfix/main.cf and /usr/local/etc/postfix/master.cf. A new installation comes with example configuration files. Move main.cf out of the way, and create a copy of master.cf:
# cd /usr/local/etc/postfix
# mv main.cf main.cf.example
# cp master.cf master.cf.example

Edit master.cf for enabling Postfix listening on the submission port 587.
# nano master.cf
Remove the hash signs in front of the blue lines:

Code:

...
#tlsproxy  unix  -       -       n       -       0       tlsproxy
[color="blue"]submission inet n       -       n       -       -       smtpd
  -o syslog_name=postfix/submission[/color]
#  -o smtpd_tls_security_level=encrypt
...

Edit the replacement file main.cf and enter the following content:
# nano main.cf

Code:

# INTERNET HOST AND DOMAIN NAMES
mydomain                            = [color="blue"]example.com[/color]
myhostname                          = mail.$mydomain
myorigin                            = $mydomain
mydestination                       = $mydomain, $myhostname, localhost, localhost.$mydomain

# REJECTING MAIL FOR UNKNOWN LOCAL USERS
local_recipient_maps                = $alias_maps, hash:/usr/local/etc/dovecot/users
unknown_local_recipient_reject_code = 550

# TRUST AND RELAY CONTROL
mynetworks                          = 127.0.0.1/32

# DELIVERY TO MAILBOX
mailbox_transport                   = lmtp:unix:private/dovecot-lmtp

# INCOMING MAIL
smtpd_sasl_auth_enable              = yes
smtpd_sasl_type                     = dovecot
smtpd_sasl_path                     = private/auth
smtpd_relay_restrictions            = permit_mynetworks,
                                      permit_sasl_authenticated,
                                      reject_unauth_destination

tls_preempt_cipherlist              = yes

smtpd_tls_security_level            = [color="blue"]encrypt[/color]
smtpd_tls_received_header           = yes
smtpd_tls_mandatory_ciphers         = high
smtpd_tls_mandatory_exclude_ciphers = aNULL, MD5
smtpd_tls_eecdh_grade               = strong
smtpd_tls_dh512_param_file          = /usr/local/etc/certs/dh_512.pem
smtpd_tls_dh1024_param_file         = /usr/local/etc/certs/dh_1024.pem
smtpd_tls_cert_file                 = /usr/local/etc/certs/mailservice.chn
smtpd_tls_key_file                  = /usr/local/etc/certs/mailservice.key

# OUTGOING MAIL
relayhost                           = [color="blue"][smtp.maildomainhoster.net]:submission[/color]
smtp_sasl_auth_enable               = yes
smtp_sasl_mechanism_filter          = digest-md5
smtp_sasl_password_maps             = hash:/usr/local/etc/postfix/relay-sasl-password

smtp_tls_security_level             = [color="blue"]encrypt[/color]
smtp_tls_mandatory_ciphers          = high
smtp_tls_mandatory_exclude_ciphers  = aNULL, MD5

The settings in blue color shall be customized.
Replace example.com by your actual domain name.
Restrict mynetworks to only localhost, i.e. force also the LAN clients to use SMTP auth for mail submission.
Utilize Dovecot for the mailbox transport.
Utilize Dovecot-SASL for SMTP client authentication.
Pass outgoing mails to the relay host of your domain & mail hosting provider.
Assign the self-signed certificate chain, and tune the TLS ciphers.
The above settings do not allow non-TLS connections. If some of you peers do not understand TLS, then you may consider to change smtpd_tls_security_level from encrypt to may.
Likewise, if the relay host does not understand TLS, set smtp_tls_security_level = may.

Anonymous · Oct 13, 2013

4. Configurations (continued)

4.4 Syslog Configuration

By default, Postfix logs everything by the way of the syslog mail facility and Dovecot has been configured to use the syslog local1 facility for logging. Without this setting, Dovecot would intermix its log messages into the same log-file as Postfix.

Edit the file /etc/syslog.conf, add the local1.* entry and change the mail.info entry to mail.* as follows:

Code:

...
local1.*                                        /var/log/dovecot.log
mail.*                                          /var/log/postfix.log
...

For configuring log rotation for the new log files, edit the file /etc/newsyslog.conf, and remove the setting for /var/log/maillog and add 2 new settings:

Code:

...
/var/log/dovecot.log                    640  7     *    @T00  JC
/var/log/postfix.log                    640  7     *    @T00  JC
...

4.5 Disable Sendmail and Start-Up Dovecot/Postfix

In file /etc/rc.conf add the following section:

Code:

###  Mail Services ###
# Disable Sendmail
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"

# Enable Dovecot POP3/IMAP
dovecot_enable="YES"

# Enable Postfix SMTP
postfix_enable="YES"

Edit the file /etc/periodic.conf, adding the following content:

Code:

daily_clean_hoststat_enable="NO"
daily_status_mail_rejects_enable="NO"
daily_status_include_submit_mailq="NO"
daily_submit_queuerun="NO"

Finally, take 3 times a deep breath - - - then:
# service sendmail stop

Code:

Stopping sendmail.
Waiting for PIDS: 1107.
sendmail_submit not running? (check /var/run/sendmail.pid).
Stopping sendmail_clientmqueue.
Waiting for PIDS: 1110.

# service dovecot start

Code:

Starting dovecot.

# service postfix start

Code:

postfix/postfix-script: starting the Postfix mail system

# ps -axj

Code:

...
root     6891    1 6891 6891    0 Is   ??     0:00.02 /usr/local/sbin/dovecot -c /usr/local/etc/dovecot/dovecot.conf
dovecot  6892 6891 6891 6891    0 I    ??     0:00.01 dovecot/anvil
root     6893 6891 6891 6891    0 I    ??     0:00.01 dovecot/log
root     6895 6891 6891 6891    0 I    ??     0:00.03 dovecot/config
...
root     7014    1 7014 7014    0 Ss   ??     0:00.05 /usr/local/libexec/postfix/master -w
postfix  7015 7014 7014 7014    0 S    ??     0:00.02 pickup -l -t unix -u
postfix  7016 7014 7014 7014    0 I    ??     0:00.02 qmgr -l -t unix -u
...

If you see more or less something like the above in your process listing, then your Home Mail Server is up and running.

Give it another clean start by rebooting the FreeBSD machine.

If the process listing does not show Dovecot and/or Postfix Daemons, then look into the log files and start with trouble shooting.
# tail -100 /var/log/dovecot.log
# tail -100 /var/log/postfix.log

Anonymous · Oct 13, 2013

5. Advanced Features

5.1 Web Mail

The typical usage scenario for a Web Mail Service on our Mail Server is to have access to our mails using foreign equipment and software, usually because our own equipment with its dedicated mail clients is not always with us, or does not work properly in all network scenarios.

For example:

at work you would use the companies computer to browse your mails using Web Mail.
on travel you use a terminal at the aiport, hotel, LAN-house, etc. because it is not guarantteed that you can connect your equipment at any time you would need it.

Being concerned on security issues, of course, we want our Web Mail to run as a HTTPS service. So the web server needs to have TLS (SSL) enabled, and we need to provide it with our certificate chain. However, enabling TLS on the server side will not always prevent 3rd party eavesdropping, and we have to be specially concerned when we are using 3rd party equipment for connecting to our Web Mail Server. Deep Packet Inspection Firewalls encrypt and re-encrypt SSL traffic in real-time, and can be configured to transfer a copy of the decrypted stream to anywhere. There are a lot of models and suppliers in the market, and the following links are not meant as a product indication, but to make you aware of the capabilities of such systems:

http://www.sonicwall.com/downloads/SonicOS_Enhanced_5.6_DPI-SSL_Feature_Module.pdf
https://realtime.demo.sonicwall.com/main.html

In the Demo, specially pay attention to DPI-SSL. The company you are working for, may have installed it, the hotel, the LAN-house, the public wireless provider may have it, perhaps the NSA, who knows? With this kind of equipment in the background, when using clients trusting the Firewalls Certificate re-signing Authority, your connection to your Web Mail Server at home appears un-encrypted to the operator of that DPI-SSL firewall.

We need to be aware of this, in order to evaluate the risks. My conclusion is, that I can live with somebody is occasionally eavesdropping me communicating to my Web Mail Account on my Home Mail Server. They would see the headers and the messages that I open. My main concern is about the user credentials of my IMAP account. The default setup of most Web Mail Clients (Roundcube among these) sends the credentials by a simple html-form, and therefore these must appear un-encrypted for the operator of a DPI-SSL firewall. Having my credentials, anybody could enter my account and read anything to his/her willing.

We have to go an extra mile for securing the credentials. So, below, Roundcube will be installed using HTTP Digest Authentication. An occasional eavesdropper would see some headers, and the messages that are opened and sent during a session, but the credentials are still safe. If you need 100 % secrecy, then don't use Web Mail with foreign equipment.

5.1.1 Installations for Roundcube Web Mail

Roundcube Web Mail requires a web server, PHP + a SQL db engine in operational state. The following assumes that working installations of Apache 2.4, PHP 5.5, and PostgreSQL 9.3 do exist already.

Install said ports and Roundcube in a row:

1) PostgreSQL (we need sudo(8)())
# cd /usr/ports/security/sudo; make install clean
# cd /usr/ports/databases/postgresql93-server; make install clean
# sudo -u pgsql /usr/local/bin/initdb -A md5 -E UTF8 -U pgsql -W -D /usr/local/pgsql/data
# chmod o+w /tmp

2) Apache + PHP
# cd /usr/ports/www/apache24; make install clean
# cd /usr/ports/lang/php5; make install clean

Create and edit the file /usr/local/etc/apache24/Includes/php5.conf containing the following:

Code:

<IfModule php5_module>
   DirectoryIndex index.php index.html
   AddType application/x-httpd-php .php
   AddType application/x-httpd-php-source .phps
</IfModule>

Add apache24 and postgresql to the rc autostart script:

Code:

...

postgresql_enable="YES"
apache24_enable="YES"

3) Install Roundcube Web Mail:
# cd /usr/ports/mail/roundcube; make install clean

Code:

+---------------------------- roundcube-0.9.5,1 -------------------------------+
| +--------------------------------------------------------------------------+ |
| |+[ ] GD      Enable GD support (image conversion)                         | |
| |+[ ] LDAP    Enable LDAP support (address book)                           | |
| |+[ ] NSC     Install network spellchecker                                 | |
| |+[ ] PSPELL  Enable PSpell support (internal spellcheck)                  | |
| |+[ ] SSL     Enable SSL support (imaps or google spellcheck)              | |
| |------------------------------------ DB ----------------------------------| |
| |+( ) MYSQL   Use MySQL backend                                            | |
| |+(*) PGSQL   Use PostgreSQL backend                                       | |
| |+( ) SQLITE  Use SQLite backend                                           | |
| +--------------------------------------------------------------------------+ |
+------------------------------------------------------------------------------+

Note: The above command will install also all the necessary PHP modules.

Finally, start apache24 and postgresql:
# service postgresql start
# service apache24 start

5.1.2 Configuration of Web Mail with Roundcube

The Web Mail service shall run on our public IP on port 443, accessible by our domain https://mail.example.com. Ideally the ISP does not block port 443, however, if it is blocked or occupied by another service, it is possible to choose another port.

1) Prepare the database for Roundcube
# createuser -U pgsql -W roundcube
# createdb -U pgsql -W -O roundcube -E UTF8 roundcubemail
# psql -U pgsql -W roundcubemail

Code:

Password for user pgsql: 
psql (9.3.1)
Type "help" for help.

roundcubemail=# ALTER USER roundcube WITH PASSWORD '[color="blue"]roundcube_db_password[/color]';
ALTER ROLE
roundcubemail=# \c - roundcube
Password for user roundcube: 
You are now connected to database "roundcubemail" as user "roundcube".
roundcubemail=> \i /usr/local/www/roundcube/SQL/postgres.initial.sql
CREATE SEQUENCE
...
CREATE TABLE
INSERT 0 1
roundcubemail=> \q

Important, you need to remember the password for the database user roundcube, roundcube_db_password.

2) Configure Apache to serve the Roundcube installation at /usr/local/www/roundcube for our Web Mail URL https://mail.example.com:

Create and edit the file /usr/local/etc/apache24/Includes/global-tls.conf containing the following:

Code:

LoadModule ssl_module           libexec/apache24/mod_ssl.so
LoadModule socache_shmcb_module libexec/apache24/mod_socache_shmcb.so
Listen 443

AddType application/x-x509-ca-cert .crt
AddType application/x-pkcs7-crl    .crl

SSLCipherSuite          TLSv1+HIGH:!3DES:!CAMELLIA:!aNULL:@STRENGTH
SSLHonorCipherOrder     on
SSLPassPhraseDialog     builtin
SSLSessionCache         "shmcb:/var/run/ssl_scache(512000)"
SSLSessionCacheTimeout  300

Create and edit the file /usr/local/etc/apache24/Includes/webmail-vhost.conf containing the following:

Code:

<VirtualHost mail.example.com:443>

   ServerName mail.[color="blue"]example.com[/color]:443
   ServerAdmin hostmaster@[color="blue"]example.com[/color]
   DocumentRoot "/usr/local/www/roundcube"
   <Directory "/usr/local/www/roundcube">
      AllowOverride All
      Require all granted
      # Require valid-user
      # AuthType Digest
      # AuthName "[color="blue"]example.com[/color]"
      # AuthDigestDomain /
      # AuthDigestProvider file
      # AuthUserFile /var/mail/users/virtusers.d/http_md5_digest
   </Directory>

   SSLEngine               on
   SSLCertificateFile      "/usr/local/etc/certs/mailservice.crt"
   SSLCertificateKeyFile   "/usr/local/etc/certs/mailservice.key"
   SSLCertificateChainFile "/usr/local/etc/certs/mailservice.chn"

   <FilesMatch "\.(cgi|shtml|phtml|php)$">
      SSLOptions +StdEnvVars
   </FilesMatch>
   <Directory "/usr/local/www/apache24/cgi-bin">
      SSLOptions +StdEnvVars
   </Directory>

   BrowserMatch "MSIE [2-5]" nokeepalive ssl-unclean-shutdown downgrade-1.0 force-response-1.0
   CustomLog    "/var/log/httpd-ssl_request.log" "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"

</VirtualHost>

Note, that the settings for HTTP Digest authentication are already placed into the virtual host configuration of Web Mail, but are still inactive, because not all pre-requisites are setup.

Restart apache24: # service apache24 restart
Now point your web browser to https://mail.example.com/installer/ (replace example.com with your actual domain). This let you:

Check the environment
Create the configuration
Test the configuration

In Step 2:

Disable spell checking -- you might not want that Google checks your mails, if you wanted this, you could have stayed with gmail in the first place.
Set the log_driver to syslog.
Set the syslog_facility to local level 1 -- i.e. the same log facility as of dovecot, and error messages would appear in that log.
Enter roundcube_db_password into the field Database password of the setting db_dsnw, i.e. the password that has been set in the course of preparing the roundcube database.
Set the default_host to localhost -- i.e. with respect to Web Mail, dovecot is running locally.
Change sent_mbox to Sent Messages -- this matches the name of the respective special directory of Apple's Mail application.
Change trash_mbox to Deleted Messages -- this matches the name of the respective special directory of Apple's Mail application.
Set the smtp_server to localhost -- i.e. with respect to Web Mail, postfix smtpd is running locally.
Leave smtp_user/smtp_pass empty and check mark "Use the current IMAP username and password for SMTP authentication".
Set mdn_requests to ignore.
Push the button <CREATE CONFIG>, and 2 generated configuration files, namely main.inc.php and db.inc.php are shown.

Download these files to your home folder, and change the following settings in ~/main.inc.php (do not replace anything outside of the blue ranges):

Code:

$rcmail_config['use_https'] = [color="blue"]true[/color];
$rcmail_config['login_lc'] = [color="blue"]0[/color];
$rcmail_config['password_charset'] = [color="blue"]'UTF-8'[/color];
$rcmail_config['default_charset'] = [color="blue"]'UTF-8';[/color]

Move said files into the configuration directory of the Roundcube installation:
# mv ~/main.inc.php /usr/local/www/roundcube/config/main.inc.php
# mv ~/db.inc.php /usr/local/www/roundcube/config/db.inc.php

Back in your web browser, go to https://mail.example.com/installer/, click on the link of step 3, Test config, and verify that everything is marked OK. If not, then you need to revise the settings.

Before going the extra mile of enabling HTTP Digest Authentication for Roundcube (s. next chapter 5.1.3), test the installation. Try to browse the directories of your IMAP account, open e-mails, write a test e-mail and send it to an external address, etc.

Once everything is working as expected, you want to delete the installer directory from the roundcube installation: # rm -r /usr/local/www/roundcube/installer.

Anonymous · Oct 13, 2013

5.1 Web Mail (continued)

5.1.3 Roundcube with HTTP Digest Authentication

Upon login on a Standard Roundcube Web Mail service, the roundcube server receives the users credentials login-id and password in clear text, and it needs the clear password for being able to locally login to the dovecot IMAP service.

The purpose of HTTP Digest Authentication is to prevent plain text password exchange over the connection, and therefore the roundcube server needs to obtain the un-encrypted password by other means from the local (server side) user/password store. In chapter 4.2.1 The Virtual User Store, we created two files, one holding the credentials for CRAM-MD5-authentication controlled by dovecot into the IMAP/POP3/SMTP services, and another one for postfix knows about its local recipients. In addition to this, for the given purpose, apache needs to be fed with just another file holding the user credentials in Digest-MD5 format, and roundcube needs to somehow lookup the un-encrypted password for a given user who logged-in by the HTTP Digest method.

With all that, manually maintaining the Virtual User Store becomes complicated, and the solution presented here, is maintaining the Virtual User information in one master file, and a program derives from that the files necessary for providing the user credentials in the required formats for the different authentication methods:

The user credentials for CRAM-MD5 authentication, i.e. imap_virtual_users.
The user credentials for Digest-MD5 authentication to the Web Mail service, i.e. http_md5_digest.
The lookup table which postfix uses to lookup the local recipients, and that the digest_authentication plugin for roundcube uses for looking up the un-encrypted password for a digest authenticated user, i.e. smtp_local_recipients.db.

The program for deriving said files from the master is vumap (Virtual User Map):

Code:

//  vumap.c
//  vumap
//
//  Compile using:
//
//    clang vumap.c -I/usr/local/include -L/usr/local/lib -Wno-parentheses -lcrypto -o vumap
//
//  Created by Dr. Rolf Jansen 2013-10-27.
//  Copyright (c) 2013. All rights reserved.
//
//  Use it for free on your own risk.

#include <stdlib.h>
#include <stdio.h>
#include <stdint.h>
#include <string.h>
#include <errno.h>
#include <unistd.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <openssl/md5.h>


typedef uint8_t MD5_BinDigest[MD5_DIGEST_LENGTH];
typedef uint8_t MD5_HexDigest[MD5_DIGEST_LENGTH*2 + 1];
typedef uint8_t HMAC_iMD5_Hex[MD5_DIGEST_LENGTH*4 + 1];

void convert2Hex(uint8_t *bin, uint8_t *hex, uint16_t len)
{
   uint8_t   c;
   uint16_t  i, j;

   for (i = 0, j = 0; i < len; i++)
   {
      c = (bin[i] >> 4) & 0xF;
      hex[j++] = (c <= 9) ? (c + '0') : (c + 'a' - 10);

      c = bin[i] & 0xF;
      hex[j++] = (c <= 9) ? (c + '0') : (c + 'a' - 10);
   }

   hex[j] = '\0';
}

void md5(const char *data, MD5_HexDigest hex)
{
   MD5_BinDigest bin;
   MD5_CTX context;
   MD5_Init(&context);
   data = (data) ?: "";
   MD5_Update(&context, (const uint8_t *)data, strlen(data));
   MD5_Final(bin, &context);

   convert2Hex(bin, hex, MD5_DIGEST_LENGTH);
}


void hmac_md5_intermediate(const char *data, HMAC_iMD5_Hex hex)
{
   MD5_CTX ctx;
   uint8_t opad[64] = {0};
   uint8_t ipad[64] = {0};
   size_t  i, n = strlen(data);

   if (n <= 64)
   {
      memcpy(opad, data, n);
      memcpy(ipad, data, n);
   }

   else
   {
      MD5_Init(&ctx);
      MD5_Update(&ctx, (const uint8_t *)data, n);
      MD5_Final(opad, &ctx);
      memcpy(ipad, opad, MD5_DIGEST_LENGTH);
   }

   for (i = 0; i < 64; i++)
   {
      opad[i] ^= 0x5c;
      ipad[i] ^= 0x36;
   }

   MD5_Init(&ctx);
   MD5_Update(&ctx, opad, 64);
   convert2Hex((uint8_t *)&ctx, hex, MD5_DIGEST_LENGTH);

   MD5_Init(&ctx);
   MD5_Update(&ctx, ipad, 64);
   convert2Hex((uint8_t *)&ctx, hex+MD5_DIGEST_LENGTH*2, MD5_DIGEST_LENGTH);
}


int main(int argc, const char *argv[])
{
   char  *mapname = NULL;

   size_t prefixLen = strlen(argv[2]);
   struct stat st;

   if ((stat(argv[2], &st) == 0 ||
       mkdir(argv[2], 0750) == 0 && stat(argv[2], &st) == 0) &&
       S_ISDIR(st.st_mode))
   {
      umask(0127);

      FILE  *tb, *dg, *vu, *lr;

      if (tb = fopen(argv[1], "r"))
      {
         char *dgname = strcat(strcpy(alloca(prefixLen+sizeof("/http_md5_digest")), argv[2]), "/http_md5_digest");
         if (dg = fopen(dgname, "w"))
         {
            char *vuname = strcat(strcpy(alloca(prefixLen+sizeof("/imap_virtual_users")), argv[2]), "/imap_virtual_users");
            if (vu = fopen(vuname, "w"))
            {
               char *lrname = strcat(strcpy(alloca(prefixLen+sizeof("/smtp_local_recipients")), argv[2]), "/smtp_local_recipients");
               if (lr = fopen(lrname, "w"))
               {
                  char *line, linebuf[4096], buffer[4096];
                  MD5_HexDigest md5hex;
                  HMAC_iMD5_Hex itmhex;

                  while (!feof(tb) && (line = fgets(linebuf, 4092, tb)))
                     // scan only lines without initial '#'
                     if (*line != '#')
                     {
                        char   c;
                        size_t i, n = strlen(line);

                        // extract the fields, the format is:
                        //
                        // login-id   plain passphrase   uid   gid
                        //

                        // strip trailing white space and cr/lf from gid
                        for (i = n - 1; i > 0 && ((c = line[i]) == ' ' || c == '\t' || c == '\r' || c == '\n'); i--);
                        line[i+1] = '\0';

                        // extract gid
                        for (; i > 0 && ((c = line[i]) != ' ' && c != '\t'); i--);
                        char *gid = line + i + 1;

                        // strip trailing white space from uid
                        for (; i > 0 && ((c = line[i]) == ' ' || c == '\t'); i--);
                        line[i+1] = '\0';

                        // extract uid
                        for (; i > 0 && ((c = line[i]) != ' ' && c != '\t'); i--);
                        char *uid = line + i + 1;

                        // strip trailing white space from the pass phrase
                        for (; i > 0 && ((c = line[i]) == ' ' || c == '\t'); i--);
                        line[n = i+1] = '\0';

                        // strip initial white space
                        for (i = 0; i < n && ((c = line[i]) == ' ' || c == '\t'); i++);
                        // extract login-id & realm
                        char *loginid = line + i;
                        char *realm   = (strchr(loginid, '@') + 1) ?: loginid;
                        for (; i < n && ((c = line[i]) != ' ' && c != '\t'); i++);
                        line[i++] = '\0';

                        // strip trailing white space from the login-id
                        for (; i < n && ((c = line[i]) == ' ' || c == '\t'); i++);
                        // extract the pass phrase
                        char *passphrase = line + i;

                        // write out the fields, by ommiting and converting as necessary
                        // the HTTP MD5-Digest
                        snprintf(buffer, 4095, "%s:%s:%s", loginid, realm, passphrase);
                        md5(buffer, md5hex);
                        fprintf(dg, "%s:%s:%s\n", loginid, realm, md5hex);

                        // the input file for the SMTP Local Recipient map (includes clear text passwords)
                        fprintf(lr, "%s %s\n", loginid, passphrase);

                        // the IMAP/POP3 virtual users file, using the joined outer+inner intermediate MD5 states as the CRAM-MD5 password hash
                        hmac_md5_intermediate(passphrase, itmhex);
                        fprintf(vu, "%s:{CRAM-MD5}%s:%s:%s:\n", loginid, itmhex, uid, gid);
                     }

                  fclose(lr);
                  mapname = lrname;
               }
               else
                  printf("Error: The Local Recipients table could not be opened for writing. %d\n", errno);

               fclose(vu);
            }
            else
               printf("Error: The IMAP/POP3 Virtual Users table could not be opened for writing. %d\n", errno);

            fclose(dg);
         }
         else
            printf("Error: The HTTP MD5 Digest file could not be opened for writing. %d\n", errno);

         fclose(tb);
      }
      else
         printf("Error: The Canonical User table could not be opened for reading. %d\n", errno);

   }
   else
      printf("Error: The target directory does not exist and could not be created. %d\n", errno);

   if (mapname)
   {
      if (fork() == 0)
         execl("/usr/local/sbin/postmap", "/usr/local/sbin/postmap", mapname, NULL);

      else
      {
         int status;
         wait(&status);
         unlink(mapname);
      }
   }

   return 0;
}

Compile it using: # clang vumap.c -I/usr/local/include -L/usr/local/lib -Wno-parentheses -lcrypto -o vumap

Create the master password file virtusers in a quite restricted directory like the home directory of the root user. You may want to use /usr/local/etc/dovecot/users that has been created in chapter 4.2.1 The Virtual User Store, as a model. Copy that file, replace the ':' by spaces or tabulators, and the {CRAM-MD5}intermediate_password_hash by the clear text pass phrase (no quotes, the password may contain spaces and any other characters). Lines starting with a '#' are ignored, for example:
# cp /usr/local/etc/dovecot/users ~/virtusers
# nano ~/virtusers

Code:

# login-id                plain passphrase             uid    gid   
rolf@example.com          password of rolf            10001  10000
suzy@example.com          password of suzy            10002  10000
alex@example.com	  password of alex            10003  10000

Add the users postfix and www to the group virtmail (s. chapter 4.2.1):
# pw groupmod virtmail -m postfix,www

Make The Mail Directory (s. chapter 4.2.2) owned by the dovecot user:
# chown dovecot /var/mail/users

Now create the new format Mail Users Store at /var/mail/users/virtusers.d:
# sudo -u dovecot vumap ~/virtusers /var/mail/users/virtusers.d

Check that the created files can be accessed by the various daemons running as the users dovecot, postfix, www.
sudo -u dovecot cat /var/mail/users/virtusers.d/imap_virtual_users

Code:

rolf@example.com:{CRAM-MD5}808388316322c028ddcec6fc1fbe2f47581a65029442e388da5e71152f799399:10001:10000:
suzy@example.com:{CRAM-MD5}5eee45c41484bfb89e5954fa8623dd81d0d916a5b76204b9daddded2f2de6115:10002:10000:
alex@example.com:{CRAM-MD5}063eb11f839746e32b470d4adc40d2130f4d3b3924ad526d78dd7605f2063c70:10003:10000:

sudo -u www cat /var/mail/users/virtusers.d/http_md5_digest

Code:

rolf@example.com:example.com:6fa5c7fbf5002c0c7ea467cbdf28e420
suzy@example.com:example.com:0bf81c0247fd457b83269848840a25c7
alex@example.com:example.com:54f7dbba2d984b7efaecf0aa81a86c77

 sudo -u postfix postmap -q [email=suzy@example.com]suzy@example.com[/email] /var/mail/users/virtusers.d/smtp_local_recipients

Code:

password of suzy

 sudo -u www postmap -q [email=alex@example.com]alex@example.com[/email] /var/mail/users/virtusers.d/smtp_local_recipients

Code:

password of alex

Note, how the master virtual users file does not make it into the new Mail Users Store. Keep the master file at a safe place, you can even remove it from the machine after mapping the users into the store.

(continued, s. next message of this How To)

Anonymous · Oct 13, 2013

5.1.3 Roundcube with HTTP Digest Authentication (continued)

Point Dovecot, Postfix and Apache to the credential files in the new Users Store.

Change the settings userdb.args and passdb.args in /usr/local/etc/dovecot/dovecot.conf to:

Code:

userdb {
  args = username_format=%u [color="blue"]/var/mail/users/virtusers.d/imap_virtual_users[/color]
  driver = passwd-file
}

passdb {
  args = username_format=%u [color="blue"]/var/mail/users/virtusers.d/imap_virtual_users[/color]
  driver = passwd-file
}

Change the setting local_recipient_maps in /usr/local/etc/postfix/main.cf to:

Code:

local_recipient_maps = $alias_maps, hash:[color="blue"]/var/mail/users/virtusers.d/smtp_local_recipients[/color]

Activate the Digest authentication in /usr/local/etc/apache24/Includes/webmail-vhost.conf, i.e. remove the line reading Require all granted and remove the hash sign '#' in front of Require valid-user and the various Auth* settings.

Code:

...
   <Directory "/usr/local/www/roundcube">
      AllowOverride All
      [color="blue"]Require valid-user
      AuthType Digest
      AuthName "example.com"
      AuthDigestDomain /
      AuthDigestProvider file
      AuthUserFile /var/mail/users/virtusers.d/http_md5_digest[/color]
   </Directory>
...

Restart Dovecot, Postfix and Apache:
# service dovecot restart
# service postfix restart
# service apache24 restart

Verify that the e-mail system is working well using the new Users Store. Try to send and receive e-mails to your account. If everything works well, then delete the old Users Store:
rm /usr/local/etc/dovecot/users
rm /usr/local/etc/dovecot/users.db

5.1.4 The Roundcube HTTP Digest Authentication Plugin

After applying the changes in the previous chapter, and entering your account via Web Mail, your browser should show its authentication dialog box, asking for your user credentials for your domain. If everything worked-out well, then the web server will let you in, however you will have to login once again, because the Roundcube authentication page is still active. We need to tell Roundcube to use the credentials from the HTTP Digest Authentication, so that it does not request another login in case the digest login was successful.

For this we need to create and activate a HTTP Digest Authentication Plugin.

1) Create the plugin directory at /usr/local/www/roundcube/plugins and 2 files in there:
# mkdir /usr/local/www/roundcube/plugins/digest_authentication
# nano /usr/local/www/roundcube/plugins/digest_authentication/digest_authentication.php

Code:

<?php

   class digest_authentication extends rcube_plugin
   {
      function init()
      {
         $this->add_hook('startup',      array($this, 'startup'));
         $this->add_hook('authenticate', array($this, 'authenticate'));
         $this->add_hook('logout_after', array($this, 'logout_after'));
      }


      function startup($args)
      {
         if (!empty($_SERVER['PHP_AUTH_USER']) && !empty($_SERVER['PHP_AUTH_DIGEST']))
         {
            $rcmail = rcmail::get_instance();
            $rcmail->add_shutdown_function(array('digest_authentication', 'shutdown'));

            if (empty($args['action']) && empty($_SESSION['user_id']))
                $args['action'] = 'login';

             else if (!empty($_SESSION['user_id']) && empty($_SESSION['password']))
                $_SESSION['password'] = $rcmail->encrypt(exec("/usr/local/sbin/postmap -q ".$_SERVER['PHP_AUTH_USER']." /var/mail/users/virtusers.d/smtp_local_recipients"));
         }

         return $args;
      }


      function authenticate($args)
      {
         if (!empty($_SERVER['PHP_AUTH_USER']) && !empty($_SERVER['PHP_AUTH_DIGEST']))
         {
            $args['user'] = $_SERVER['PHP_AUTH_USER'];
            $args['pass'] = exec("/usr/local/sbin/postmap -q ".$_SERVER['PHP_AUTH_USER']." /var/mail/users/virtusers.d/smtp_local_recipients");
         }

         $args['cookiecheck'] = false;
         $args['valid'] = true;

         return $args;
      }


      function logout_after($args)
      {
         if (!empty($_SERVER['PHP_AUTH_USER']) && $args['user'] == $_SERVER['PHP_AUTH_USER'])
            header('Location: plugins/digest_authentication/logout.php');
      }


      function shutdown()
      {
         rcmail::get_instance()->session->remove('password');
      }
   }

?>

Note, how the postmap(1)() utility that came with Postfix is employed to lookup the plain text password in the smtp_local_recipients lookup table of the user who has been already authenticated by his digest.

# nano /usr/local/www/roundcube/plugins/digest_authentication/logout.php

Code:

<?PHP
   if (isset($_GET['authenticate']))
   {
      header('HTTP/1.1 401 Unauthorized');
      header('WWW-Authenticate: Digest realm="example.com", qop="auth", nonce="Log me out."');
   }
?>

<!DOCTYPE html>
<HTML>
<HEAD>
   <META http-equiv="Content-Type" content="text/html; charset=utf-8">
   <META http-equiv="refresh" content="1; URL=logout.php?authenticate">
   <TITLE>Webmail Logging out!</TITLE>
</HEAD>
<BODY>
<H2>Logging out!</H2>
</BODY>
</HTML>

2) Activate the plugin in the main configuration file /usr/local/www/roundcube/config/main.inc.php of Roundcube, by finding the setting $rcmail_config['plugins'] and changing it to:

Code:

$rcmail_config['plugins'] = array([color="blue"]'digest_authentication'[/color]);

3) Quit your browser (so it forgets the authentication session), restart it, and login once again into your Web Mail. This time, after effective digest authentication, Roundcube shouldn't show its own login page anymore, but let you directly to your mails.

Finally, there you are, and your password did not go in plain-text through the line. Happy Web-Mailing!

Anonymous · Oct 13, 2013

5.2 Blocking Spam

5.2.1 Before queue validation

5.2.2 Enabling Grey Listing

5.2.3 Statistical Spam Filter using CRM114 (eventually before queue)

5.3 Try encrypted direct delivery first, and use the SMTP relay only as fallback

Anonymous · Oct 13, 2013

6. Wish list for the Future

6.1 Digest-SHA3 authentication scheme for Web and Mail

6.2 Autonomous X.509 PKI

6.3 Direct End-to-end delivery + TLS

Anonymous · Oct 13, 2013

6. Wish list for the Future (continued)

6.4 ...

Anonymous · Oct 13, 2013

(placeholder)

throAU · Oct 22, 2013

Awesome, I was looking to do something similar, but a lot more "cookie cutter" for a couple of basic scenarios (i.e., set up for local mail delivery to the postfix box, or acting as an edge server for relay to an internal host). I'm by no means an SMTP novice, but its been quite a few years since I've put any serious effort into revising the configuration with basic anti-relay that I originally started running about 10 years ago (occupied with other things like VOIP and virtualization).

I'm really keen to get anti-spam working on postfix for the second scenario (relay to internal destination) above, all the anti-spam documentation I've seen seems to assume you are delivering mail to the local postfix/sendmail machine, rather than relaying to another host. Could I request that as a "wish list" item? I'll be more than willing to help test that particular scenario, I currently have a test-lab network set up that I'm about to use for such purposes.

I'll certainly be following this thread!

Anonymous · Oct 22, 2013

throAU said:
I'm really keen to get anti-spam working on postfix for the second scenario (relay to internal destination) above, all the anti-spam documentation I've seen seems to assume you are delivering mail to the local postfix/sendmail machine, rather than relaying to another host. Could I request that as a "wish list" item? I'll be more than willing to help test that particular scenario, I currently have a test-lab network set up that I'm about to use for such purposes.

I guess you are talking about a SMTP server at the un-trusted/trusted border that receives mail for a variety of other servers and does already the spam checking/blocking. If not, then please explain a little bit more the desired setup.

OK, strictly technical speaking, this Postfix server is on said border, only that at the trusted side there is no delivery to another SMTP host but local delivery, i.e. LMTP. However, with respect to the planned chapter 5.2 Spam Blocking, this would make no difference. The Postfix terminology "Before Queue" means that something happens already in the course of receiving a message, and before it is queued for delivery, either of SMTP or LMTP.

IMHO, ideally Spam Blocking happens in the "Before Queue" stage, because the connection to the sending peer would be still open, and instead of 250 OK, we can send 550 Spam blocked, and undesired messages wouldn't even make it into the trusted network behind the border. Traditional "Before Queue" spam checking includes:

HELO sanity checks
Header checks
DNS validation and checks
Public Mailer sanity
DNSRBL's and/or Grey-Listing
other more or less effective voodoo

Usually, statistical content analyses and filtering happens "After Queue", i.e. the message entered already our responsibility, and if we finally decide not to accept it, there is no way anymore to revoke our initial 250 OK. Therefore, it would be nice to do also the statistical filtering before the queue. I want to elaborate on this, and on how to eliminate the risk of loosing false positives.

I guess, that all this could be easily adapted also to a relaying SMTP server, and perhaps another topic on the wish list is not necessary. Anyway, I am open to put that and other topics on there, however, please elaborate on your wishes a little bit.

throAU · Oct 23, 2013

rolfheinrich said:
I guess you are talking about a SMTP server at the un-trusted/trusted border that receives mail for a variety of other servers and does already the spam checking/blocking. If not, then please explain a little bit more the desired setup.

Exactly. I'm currently actually doing this with sendmail in the DMZ but I'd like to migrate to postfix due to less insane configuration files, better security, etc.

The issues I'm running into in particular with sendmail are with regards to spamassassin use in the case of non-local mail delivery (I had it sort of working long ago, but all the documentation I've seen seems focused on local delivery), and reliably checking a user exists before relaying to the internal server (to prevent relay and then bounce back-scatter).

I have been doing this by importing a list of valid exchange addresses and adding this list to the access_db periodically, however for some wierd/unknown reason when I do this it breaks mail for a few remote domains that are doing sender address verification. Unfortunately the remote domains are business critical and do not have IT staff available to help resolve.

Essentially I have an internal Exchange server (this will not be going away, internally we're largely a Microsoft shop) that I do not wish to expose directly to the internet, and plan to run some sort of non-windows based filtering relay in the DMZ in front of it. Whether it ends up being FreeBSD (preferably) or an Ironport, or whatever.

For reducing spam, I'm currently doing the following (which may also be useful for your guide for non-relay use also):

zen blocklist
throttling connections
max recipients per message of 10 (more recipients will come through as multiple messages - spammers won't split the message and re-try)
greeting pause of 5 seconds before response (tarpitting)
greylisting via smf-grey milter

spamassassin is not currently in use and I'm relying on internal Exchange filtering from that point. However the above appears to block well over 90% of spam before it even gets relayed further or subject to additional filtering.

Anonymous · Oct 24, 2013

throAU said:
Exactly. I'm currently actually doing this with sendmail in the DMZ but I'd like to migrate to postfix due to less insane configuration files, better security, etc.

The issues I'm running into in particular with sendmail are with regards to spamassassin use in the case of non-local mail delivery (I had it sort of working long ago, but all the documentation I've seen seems focused on local delivery), and reliably checking a user exists before relaying to the internal server (to prevent relay and then bounce back-scatter).

I have been doing this by importing a list of valid exchange addresses and adding this list to the access_db periodically, however for some wierd/unknown reason when I do this it breaks mail for a few remote domains that are doing sender address verification. Unfortunately the remote domains are business critical and do not have IT staff available to help resolve.

With Postfix you would also export the list of valid users and use postmap to create a hash table for the local_recipient_maps setting, see chapter 4.3.3 The Postfix Configuration Files of this How To.

throAU said:
Essentially I have an internal Exchange server (this will not be going away, internally we're largely a Microsoft shop) that I do not wish to expose directly to the internet, and plan to run some sort of non-windows based filtering relay in the DMZ in front of it. Whether it ends up being FreeBSD (preferably) or an Ironport, or whatever.

In the setup elaborated above, Postfix delegates the local delivery and SMTP authentication to Dovecot. For this you would need to find out how to do this with MS Exchange.

throAU said:
For reducing spam, I'm currently doing the following (which may also be useful for your guide for non-relay use also):

zen blocklist

throttling connections

max recipients per message of 10 (more recipients will come through as multiple messages - spammers won't split the message and re-try)

greeting pause of 5 seconds before response (tarpitting)

greylisting via smf-grey milter

spamassassin is not currently in use and I'm relying on internal Exchange filtering from that point. However the above appears to block well over 90% of spam before it even gets relayed further or subject to additional filtering.

I certainly will consider your list. Some years ago, I was responsible for a number of small mail servers, and I can confirm that "Before Queue" sanity and spam checks prevent already 90 % of the dirt from coming in. The rest was effectively cleaned-up by a strictly TOE trained CRM114 -- wrong classification was approx. 1 of 2000, i.e. 0.025 % false positives and 0.025 % false negatives. So, the overall anti-dirt efficiency was 99.9975 % (90 % "Before Queue" + 9.9975 % "After Queue"), say, only 1 of 40000 spams made it into the inbox of a user.

I am going to set up a modern library based CRM114 for my Home Mail Server, and after some time of training "After Queue", I am keen to see it working "Before Queue".

I had very bad experiences with SpamAssassin at that time, and I am keeping on to like only its name, to say the least.

xy16644 · Dec 12, 2013

Re: HOWTO: Home Mail Server with TLS and non-Plain authentic

I have always used PLAIN authentication with SSL/TLS and never thought about DPI firewalls intercepting my traffic/SSL sessions! Thanks for the great post.

You said you chose the CRAM-MD5 authentication mechanism since it was compatible with most clients. How does one find out what authentication mechanisms your client(s) support? I did a quick search online but came up empty handed. I use Thunderbird and K9 mail (on Android) but I also have one Outlook 2010 user.

I was looking at SCRAM-SHA-1 which seems much newer and more secure but is support lacking in email clients? Its been around since 2010 which makes me wonder why its not supported yet in modern email clients.

HOWTO: Home Mail Server with TLS and non-Plain authenticatio

Anonymous

Guest

Anonymous

Guest

Attachments

Anonymous

Guest

Anonymous

Guest

Anonymous

Guest

Anonymous

Guest

Anonymous

Guest

Anonymous

Guest

Anonymous

Guest

Anonymous

Guest

Anonymous

Guest

Anonymous

Guest

Anonymous

Guest

throAU

Anonymous

Guest

throAU

Anonymous

Guest

xy16644