Cheap VPS & Xen Server

Residential Proxy Network - Hourly & Monthly Packages

Drupal + Postfix Integration Under Ubuntu 8.04 (Hardy)


This document is derived from Christoph Haas’ tutorial “Howto: ISP-style Email Server with Debian-Etch and Postfix” (http://workaround.org) and Falko Timme’s tutorial “Virtual Users And Domains With Postfix, Courier, MySQL And SquirrelMail (Ubuntu 8.04 LTS)” (http://www.Kreationnext.com). The associated Mailfix Drupal module must be installed as part of this tutorial.

It will guide you through the necessary steps to configure a Drupal driven Mail server. Provided features:

  • Drupal managed email accounts
  • Support for virtual domains
  • Automatic forwarding
  • Postfix quota support
  • Silent BCC monitoring
  • Other features: anti-spam, anti-virus

 

Preface

This tutorial is the result of many attempts to replace a couple of Postfix mail servers which were maintained with text files. Every time I had to create or drop an account I was required to vi those files, then reload the Postfix configuration. Same thing when users went on vacation (automatic forwarding) or supervisors asked for BCC monitoring. Yep, not funny.

Then Drupal was introduced to our corporate environment to manage our Intranet. The result was a total success. I thought how great it could be if we could manage our mail users from this same location.

So I went through such superb works from Christoph Haas (http://workaround.org/articles/ispmail-etch/) and Falko Timme (http://www.Kreationnext.com/virtual-users-domains-postfix-courier-mysql-squirrelmail-ubuntu8.04).

Then I found a great Drupal module from Thomas Barregren (http://drupal.org/project/mailserver) which had almost all the functionality we required, and was based on Christoph Haas’ work for the Postfix/MySQL integration. We thoroughly tested and reviewed this module for a couple of weeks. Unfortunately we still had to maintain the mail quota, forwarding and BCC monitoring part by hand.
Inspired by such awesome work, we finally decided to write a Drupal 6 module in order to keep all things in one place.

This is how this tutorial and its associated Drupal Mailfix module came to life.

 

Preliminary Tasks

First of all you need to install Linux on your target machine. We installed Linux Ubuntu 8.04 Server.

Select the following software packages:

  • OpenSSH server

Ensure that your IP will always be the same (fixed IP, or DHCP + IP reservation).
Write down your installation credentials:

Credentials Sample Settings
Linux user: someuser
password: opensesame
MySQL user: root@localhost
password: opensesameDrupal database: drupal_example_com
user: drupal_example_com@localhost
password: opensesame
Drupal user: admin
password: opensesame

Once your system makes a fresh boot to the newly installed OS, all tasks will require root privileges. Thus you should start a root session right away:

sudo -i

Install All Required Packages

You can use either apt-get or aptitude. We prefer aptitude. First of all update your repository tree (“u” command). Then ensure that the following packages are installed:

amavisd-new apache2 build-essential cabextract clamav-daemon debhelper dovecot-imapd dovecot-pop3d dpatch elinks fakeroot libcdb-dev libclass-dbi-mysql-perl phpmyadmin libapache2-mod-php5 libdb-dev libgdbm-dev libldap2-dev libmysqlclient15-dev libpam-mysql libpam-smbpass libpcre3-dev libpq-dev libsasl2-2 libsasl2-modules libsasl2-modules-sql libsasl2-dev libssl-dev mutt mysql-server nomarch openssl pax php5 php5-dev php5-mysql php-pear postfix postfix-doc postfix-mysql sasl2-bin spamassassin squirrelmail squirrelmail-decode vim vim-runtime

When asked about configuration options proceed as follows (replace example.com with appropriate domain):

MySQL root password: opensesame
Postfix Configuration: Internet Site
System Mail Name: example.com
phpMyAdmin Configuration: apache2

If you are a vim user, you may want to enable vim syntax highlighting by uncommenting the syntax on line in /etc/vim/vimrc configuration file:

vim /etc/vim/vimrc

  syntax on

 

Apply the Quota Patch to Postfix

We need to download Postfix sources, apply the quota patch, build new .deb packages and install them.

cd /usr/src
apt-get source postfix

Now we download the Postfix Virtual Delivery Agents (VDA) patches by Anderson Nadal (http://vda.sourceforge.net/). Keep sure you are downloading the appropriate version for your server. If in doubt, the following command will display your Postfix version:

postconf | grep ^mail_version

 mail_version = 2.5.1

In our case we download the patch for version 2.5.1:

wget http://vda.sourceforge.net/VDA/postfix-2.5.1-vda-ng.patch.gz
gunzip postfix-2.5.1-vda-ng.patch.gz

This should unpack the postfix-2.5.1-vda-ng.patch file which must be applied in order to have support for user quota.

cd postfix-2.5.1
patch -p1 < ../postfix-2.5.1-vda-ng.patch
dpkg-buildpackage

If you see the following message once the compilation ends, don’t worry, it’s as expected:

dpkg-buildpackage: warning: Failed to sign .dsc and .changes file

Now move to the upper directory, where the freshly genereated .deb files reside:

cd ..
ls -1 *.deb

postfix_2.5.1-2ubuntu1.1_i386.deb
postfix-cdb_2.5.1-2ubuntu1.1_i386.deb
postfix-dev_2.5.1-2ubuntu1.1_all.deb
postfix-doc_2.5.1-2ubuntu1.1_all.deb
postfix-ldap_2.5.1-2ubuntu1.1_i386.deb
postfix-mysql_2.5.1-2ubuntu1.1_i386.deb
postfix-pcre_2.5.1-2ubuntu1.1_i386.deb
postfix-pgsql_2.5.1-2ubuntu1.1_i386.deb

(The actual versions of your compiled packages may differ slightly.)

Then we install patched postfix and postfix-mysql packages:

dpkg -i postfix_2.5.1-2ubuntu1.1_i386.deb
dpkg -i postfix-mysql_2.5.1-2ubuntu1.1_i386.deb

Apache Configuration

/etc/hosts should have the following line:

127.0.0.1 localhost  myserver.example.com

The web server configuration is mainly a matter of taste. In this tutorial we explain how to setup a Drupal multi-hosted site. The domain example.com will have the application files under /var/www/example.com, a SSL application root under /var/ssl/example.com and the Drupal uploaded files will be stored in private mode under /var/files/example.com. The latter directory will be created when we unpack and install Drupal. This is the reason why the following commands will not create this directory right now.

Create the http folders under /var/www and https folders under /var/ssl:

mkdir /var/www/html
mv /var/www/index.html /var/www/html/
mkdir -p /var/ssl/example.com

The /var/www/html folder will contain default web configuration. For example, if you access to the host using IP address, Apache will serve files contained in this directory.

The /var/ssl folder will contain sites for SSL connections.

At the bottom of /etc/apache2/apache2.conf include the following lines:

NameVirtualHost *:80
NameVirtualHost *:443
ServerName myserver.example.com

The default configuration file /etc/apache2/sites-available/default:

vim /etc/apache2/sites-available/default

<VirtualHost *:80>
  ServerAdmin webmaster@localhost
  DocumentRoot /var/www/html/
  <Directory />
    Options FollowSymLinks
    AllowOverride None
  </Directory>
  <Directory /var/www/html/>
    Options -Indexes FollowSymLinks MultiViews
    AllowOverride None
    Order allow,deny
    allow from all
  </Directory>
  ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
  <Directory "/usr/lib/cgi-bin">
    AllowOverride None
    Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
    Order allow,deny
    Allow from all
  </Directory>
  ErrorLog /var/log/apache2/error.log
  # Possible values include: debug, info, notice, warn, error, crit,
  # alert, emerg.
  LogLevel warn
  CustomLog /var/log/apache2/access.log combined
  ServerSignature On
    Alias /doc/ "/usr/share/doc/"
    <Directory "/usr/share/doc/">
        Options Indexes MultiViews FollowSymLinks
        AllowOverride None
        Order deny,allow
        Deny from all
        Allow from 127.0.0.0/255.0.0.0 ::1/128
    </Directory>
</VirtualHost>

The example.com configuration file /etc/apache2/sites-available/example.com:

vim /etc/apache2/sites-available/example.com

<VirtualHost *:80>
  ServerAdmin admin@example.com
  ServerName example.com
  ServerAlias www.example.com
  DirectoryIndex index.php
  DocumentRoot /var/www/example.com/
  <Directory />
    Options FollowSymLinks
    AllowOverride None
  </Directory>
  <Directory /var/www/example.com/>
    Options -Indexes FollowSymLinks MultiViews
    AllowOverride None
    Order allow,deny
    allow from all
  </Directory>
  ScriptAlias /cgi-bin/ /var/www/example.com/cgi-bin/
  <Directory "/var/www/example.com/cgi-bin">
    AllowOverride None
    Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
    Order allow,deny
    Allow from all
  </Directory>
  ErrorLog /var/log/apache2/example.com.error.log
  CustomLog /var/log/apache2/example.com.access.log combined
  LogLevel warn
  ServerSignature On
</VirtualHost>
<VirtualHost *:443>
  ServerAdmin admin@example.com
  ServerName example.com
  ServerAlias www.example.com
  DirectoryIndex index.html
  DocumentRoot /var/ssl/example.com/
  <Directory />
    Options FollowSymLinks
    AllowOverride None
  </Directory>
  <Directory /var/ssl/example.com/>
    Options Indexes FollowSymLinks MultiViews
    AllowOverride None
    Order allow,deny
    allow from all
  </Directory>
  ScriptAlias /cgi-bin/ /var/ssl/example.com/cgi-bin/
  <Directory "/var/ssl/example.com/cgi-bin">
    AllowOverride None
    Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
    Order allow,deny
    Allow from all
  </Directory>
  ErrorLog /var/log/apache2/ssl.example.com.error.log
  CustomLog /var/log/apache2/ssl.example.com.access.log combined
  LogLevel warn
  ServerSignature On
</VirtualHost>

Now we enable both sites and apply the configuration:

cd /etc/apache2/sites-enabled/
ln –s ../sites-available/default 000-default
ln –s ../sites-available/example.com 001-example.com
/etc/init.d/apache2 restart

Drupal Installation and Configuration

We will download and install Drupal 6.x. Then we will download and install Mailfix module (http://drupal.org/project/mailfix) which as of this writing is only available for Drupal version 6.x.

Drupal files will be unpacked in /var/www/example.com.

cd /var/www
wget http://ftp.drupal.org/files/projects/drupal-6.x.tar.gz

As of this writing, current version is 6.4, thus the target file would be drupal-6.4.tar.gz. Just keep in mind that you must replace the 6.x part with the appropriate Drupal version.

tar -xvzf drupal-6.x.tar.gz
mv drupal-6.x example.com
cd example.com
mkdir -p sites/all/modules
cd sites/all/modules
wget http://ftp.drupal.org/files/projects/mailfix-6.x-1.1.tar.gz

tar -xvzf mailfix-6.x-1.1.tar.gz
cd /var/www
chown -R www-data.www-data example.com

Now we need to create the Drupal database, which will be named as drupal_example_com in our tutorial and will be accessible with full privileges by user drupal_example_com@localhost:

mysql -p
mysql> create database drupal_example_com;

Query OK, 1 row affected (0.00 sec)

mysql> grant all privileges on drupal_example_com.* to drupal_example_com @localhost identified by ‘secret’ with grant option;

Query OK, 0 rows affected (0.00 sec)

mysql> quit

Web interface configuration can proceed by accessing http://www.example.com directly. Follow the instructions on screen to setup your Drupal site.

The next task to accomplish is to create your first Drupal account, which will be granted total administrative privileges. Remember we asked you to write down your Drupal credentials before? Well, this is where you’re expected to apply them.

Then we configure cron entry for Drupal self-maintenance:

crontab -e

# Example Drupal site cron
49 * * * * /usr/bin/wget -O - -q http://www.example.com/cron.php &> /dev/null

The &> /dev/null part is intended to reduce mail alerts issued by crontab.

Now we configure uploaded files directory. This is also a matter of personal taste. We prefer those files to be transferred by Drupal, so they must reside outside the /var/www/example.com directory. We create /var/drupal_files directory for this purpose and make it writable by Apache daemon:

cd /var
mkdir -p drupal_files/example.com
chown -R www-data.www-data drupal_files/

Inside Drupal we configure the File System with this structure in mind:

  • File System Path: /var/drupal_files/example.com
  • Download Method: Private

You may proceed to install and configure the Mailfix module:

  • Go to Administer ? Site building ? Modules
  • Enable the Mailfix module

From here you may add all mail domains your Drupal system is intended to control:

  • Go to Administer ? Site Configuration ? Mailfix domains
  • Then go to Add domains

The Mailfix documentation is self-explanatory.

Postfix Configuration

We create a user and group called vmail with uid and gid set as 5000 with the home directory /home/vmail. This is where all mail boxes will be stored.

groupadd -g 5000 vmail
useradd -g vmail -u 5000 vmail -d /home/vmail -m

Now is the time to configure Postfix directives.

The drupal-domains.cf tells Postfix which are the domains managed by your Drupal installation:

vim /etc/postfix/drupal-domains.cf

user = drupal_example_com
password = opensesame
dbname = drupal_example_com
query = SELECT domain_name AS domain FROM mailfix_domains
  WHERE domain_name = '%s'
hosts = 127.0.0.1

Only active Drupal accounts (users.status = 1) will receive emails. You can still configure disabled accounts to forward their mails to another destination by means of mailfix_users.forward field.

vim /etc/postfix/drupal-mailboxes.cf

user = drupal_example_com
password = opensesame
dbname = drupal_example_com
query = SELECT CONCAT(md.domain_name, '/', LEFT(u.mail, LOCATE('@', u.mail) - 1),'/') AS maildir
  FROM mailfix_domains md JOIN (mailfix_users mu JOIN users u
  ON mu.uid = u.uid)
  ON md.domain_id = mu.domain_id
  WHERE u.status = 1 AND u.mail ='%s'
hosts = 127.0.0.1

Forwarded mails apply to both active or disabled Drupal accounts. Some users could simply be on termporary leave or vacations (users.status = 1) while other users may correspond to off-boarding scenarios (disabled accounts, i.e. users.status = 0). Both situations can be handled with automatic mail forwarding.

vim /etc/postfix/drupal-forward.cf

user = drupal_example_com
password = opensesame
dbname = drupal_example_com
query = SELECT mu.forward
  FROM mailfix_users mu JOIN users u
  ON mu.uid = u.uid
  WHERE u.mail = '%s' AND LENGTH(mu.forward) > 0
hosts = 127.0.0.1

Postfix silent BCC monitoring is split in two parts: incoming and outgoing mails monitoring.

vim /etc/postfix/drupal-recipient-bcc.cf

user = drupal_example_com
password = opensesame
dbname = drupal_example_com
query = SELECT mu.incoming_bcc
  FROM mailfix_users mu JOIN users u
  ON mu.uid = u.uid
  WHERE u.status = 1 AND u.mail = '%s' AND LENGTH(mu.incoming_bcc) > 0
hosts = 127.0.0.1

vim /etc/postfix/drupal-sender-bcc.cf

user = drupal_example_com
password = opensesame
dbname = drupal_example_com
query = SELECT mu.outgoing_bcc
  FROM mailfix_users mu JOIN users u
  ON mu.uid = u.uid
  WHERE u.status = 1 AND u.mail = '%s' AND LENGTH(mu.outgoing_bcc) > 0
hosts = 127.0.0.1

Quota management.

vim /etc/postfix/drupal-quota.cf

user = drupal_example_com
password = opensesame
dbname = drupal_example_com
query = SELECT mu.quota
  FROM mailfix_users mu JOIN users u
  ON mu.uid = u.uid
  WHERE  u.mail = '%s'
hosts = 127.0.0.1

Then we must ensure MD5 password encryption is being used. For this we modify /etc/pam.d/smtp mapping to Drupal users table:

vim /etc/pam.d/smtp

auth    required   pam_mysql.so user=drupal_example_com passwd=opensesame host=127.0.0.1 db=drupal_example_com table=users usercolumn=mail passwdcolumn=pass crypt=0
account sufficient pam_mysql.so user=drupal_example_com passwd=opensesame host=127.0.0.1 db=drupal_example_com table=users usercolumn=mail passwdcolumn=pass crypt=0

(Parameter crypt=1 seems to be for ENCRYPT function, and crypt=0 seems to be for MD5 function.)

Then we need to restart Postfix and Saslauthd:

/etc/init.d/postfix restart
/etc/init.d/saslauthd restart

Then modify /etc/postfix/sasl/smtpd.conf:

vim /etc/postfix/sasl/smtpd.conf

pwcheck_method: saslauthd
mech_list: plain login
allow_plaintext: true
auxprop_plugin: mysql
sql_hostnames: 127.0.0.1
sql_user: drupal_example_com
sql_passwd: opensesame
sql_database: drupal_example_com
sql_select: SELECT pass FROM users WHERE mail = '%u' AND status = 1

Now apply required directives to make Postfix map to all these files:

postconf -e ‘myhostname = server1.example.com’
postconf -e ‘mydestination = server1.example.com, localhost, localhost.localdomain’
postconf -e ‘mynetworks = 127.0.0.0/8’
postconf -e ‘virtual_alias_domains =’
postconf -e ‘virtual_alias_maps = proxy:mysql:/etc/postfix/drupal-forward.cf’
postconf -e ‘virtual_mailbox_domains = proxy:mysql:/etc/postfix/drupal-domains.cf’
postconf -e ‘virtual_mailbox_maps = proxy:mysql:/etc/postfix/drupal-mailboxes.cf’
postconf -e ‘virtual_mailbox_base = /home/vmail’
postconf -e ‘virtual_uid_maps = static:5000’
postconf -e ‘virtual_gid_maps = static:5000’
postconf -e ‘smtpd_sasl_auth_enable = yes’
postconf -e ‘broken_sasl_auth_clients = yes’
postconf -e ‘smtpd_sasl_authenticated_header = yes’
postconf -e smtpd_sasl_path=private/auth
postconf -e smtpd_sasl_type=dovecot
postconf -e ‘smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination’
postconf -e ‘smtpd_use_tls = yes’
postconf -e ‘smtpd_tls_cert_file = /etc/postfix/smtpd.cert’
postconf -e ‘smtpd_tls_key_file = /etc/postfix/smtpd.key’
postconf -e ‘virtual_create_maildirsize = yes’
postconf -e ‘virtual_mailbox_extended = yes’
postconf -e ‘virtual_mailbox_limit_maps = proxy:mysql:/etc/postfix/drupal-quota.cf’
postconf -e ‘virtual_mailbox_limit_override = yes’
postconf -e ‘virtual_maildir_limit_message = “The user you are trying to reach is over quota.”‘
postconf -e ‘virtual_overquota_bounce = yes’
postconf -e ‘recipient_bcc_maps = proxy:mysql:/etc/postfix/drupal-recipient-bcc.cf’
postconf -e ‘sender_bcc_maps = proxy:mysql:/etc/postfix/drupal-sender-bcc.cf’
postconf -e ‘proxy_read_maps = $local_recipient_maps $mydestination $virtual_alias_maps $virtual_alias_domains $virtual_mailbox_maps $virtual_mailbox_domains $relay_recipient_maps $relay_domains $canonical_maps $sender_canonical_maps $recipient_canonical_maps $relocated_maps $transport_maps $mynetworks $virtual_mailbox_limit_maps $recipient_bcc_maps $sender_bcc_maps’

The smtpd_tls_cert_file and smtpd_tls_key_file directives for SSL client connectivity have been setup. However the corresponding files do not exist yet. The following commands will create them (replace highlighted settings with your own):

cd /etc/postfix
openssl req -new -outform PEM -out smtpd.cert -newkey rsa:2048 -nodes -keyout smtpd.key -keyform PEM -days 365 -x509

Country Name (2 letter code) [AU]: BO
State or Province Name (full name) [Some-State]: La Paz
Locality Name (eg, city) []: La Paz
Organization Name (eg, company) [Internet Widgits Pty Ltd]: My Company
Organizational Unit Name (eg, section) []: IT
Common Name (eg, YOUR name) []: www.example.com
Email Address []: john@example.com

For security reasons it is adviseable that you change the permissions for smtpd.key:

chmod o= /etc/postfix/smtpd.key

Deliver Incoming Mails Through the Dovecot LDA

To make Postfix use Dovecot you need to append one line to /etc/postfix/master.cf:

vim /etc/postfix/master.cf

dovecot   unix  -       n       n       -       -       pipe
  flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${recipient}

Reload Postfix:

/etc/init.d/postfix reload

Now make Postfix deliver mails through this service:

postconf -e virtual_transport=dovecot
postconf -e dovecot_destination_recipient_limit=1

 

Configure Dovecot

Dovecot supports PO3, POP3S, IMAP and IMAPS. To enable all these services modify /etc/dovecot/dovecot.conf to match the following settings:

vim /etc/dovecot/dovecot.conf

protocols = imap imaps pop3 pop3s
mail_location = maildir:/home/vmail/%d/%n/Maildir
auth default {
  mechanisms = plain login
  passdb sql {
    args = /etc/dovecot/dovecot-sql.conf
  }
  userdb static {
    args = uid=5000 gid=5000 home=/home/vmail/%d/%n allow_all_users=yes
  }
  socket listen {
    master {
      path = /var/run/dovecot/auth-master
      mode = 0600
      user = vmail
    }
    client {
      path = /var/spool/postfix/private/auth
      mode = 0600
      user = postfix
      group = postfix
    }
  }
}
protocol lda {
  log_path = /home/vmail/dovecot-deliver.log
  global_script_path = /home/vmail/globalsieverc
  postmaster_address = postmaster@example.com
  auth_socket_path = /var/run/dovecot/auth-master
  mail_plugins = cmusieve
}

Edit /etc/dovecot/dovecot-sql.conf and make sure the following directives are set:

vim /etc/dovecot/dovecot-sql.conf

driver = mysql
connect = host=127.0.0.1 dbname=drupal_example_com user=drupal_example_com password=opensesame
default_pass_scheme = PLAIN-MD5
password_query = SELECT mail AS user, pass AS password FROM users WHERE mail='%u';

Then you must restart Dovecot:

/etc/init.d/dovecot restart

That’s it! Now Drupal and Postfix share the same authentication repository.

 

References

  1. Falko Timme, © 2008, Virtual Users And Domains With Postfix, Courier, MySQL And SquirrelMail (Ubuntu 8.04 LTS), http://www.Kreationnext.com/virtual-users-domains-postfix-courier-mysql-squirrelmail-ubuntu8.04
  2. Christoph Haas, © 2007, Howto: ISP-style Email Server with Debian-Etch and Postfix, http://workaround.org/articles/ispmail-etch/
  3. Matt Butcher, Learning Drupal 6 Module Development, ISBN 190-4811-80-9, © 2008 Packt Publishing
  4. Thomas Barregren, Mailserver Drupal Module, http://drupal.org/project/mailserver
  5. Alex Saavedra, © 2008, Mailfix Drupal Module, http://drupal.org/project/mailfix

Comments

comments