Cheap VPS & Xen Server

Residential Proxy Network - Hourly & Monthly Packages

The Perfect SpamSnake – Ubuntu Jaunty Jackalope


This tutorial shows how to set up an Ubuntu Jaunty Jackalope based server as a spamfilter in Gateway mode. In the end, you will have a SpamSnake Gateway which will relay clean emails to your MTA. You will also be able to view your incoming queue, train your SpamSnake and carry out a few more advanced operations via MailWatch.

I cannot offer any guarantees that this will work for you, the same way it’s working for me.

I will use the following software:

  • Web Server: Apache 2 with PHP 5
  • Database Server: MySQL 5.0
  • Mail Server: Postfix
  • Caching DNS Server: Dnsmasq
  • MailScanner: MailScanner v4.76
  • MailWatch: MailWatch v1.0.4

Credit goes to the guys at Kreationnext and the developers of MailScanner, MailWatch, ClamAV, Apache, Mysql and Postfix.

 

Install the base system using the minimal option.

1. Get root Privileges

Enable the root login by running the following and giving root a password. You can then directly log in as root:

sudo passwd root

 

 2. Install vim-nox (Optional)

I’ll use vi as my text editor in this tutorial. The default vi program has some strange behaviour on Ubuntu and Debian; to fix this, we install vim-nox:

aptitude install vim-nox

(You don’t have to do this if you use a different text editor such as joe or nano.)

 

 3. Configure The Network

Because the Ubuntu installer has configured our system to get its network settings via DHCP, we have to change that now because a server should have a static IP address. Edit /etc/network/interfaces and adjust it to your needs (in this example setup I will use the IP address 192.168.0.100):

vi /etc/network/interfaces

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
        address 192.168.0.100
        netmask 255.255.255.0
        network 192.168.0.0
        broadcast 192.168.0.255
        gateway 192.168.0.1

Then restart your network:

/etc/init.d/networking restart

Then edit /etc/hosts. Make it look like this:

127.0.0.1       localhost.localdomain   localhost
192.168.0.100   server1.example.com     server1

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

Now run:

echo server1.example.com > /etc/hostname
/etc/init.d/hostname.sh start

Afterwards, run:

hostname
hostname -f

Both should show server1.example.com now.

 

4. Update your Linux Installation

Edit /etc/apt/sources.list. Comment out or remove the installation CD from the file and make sure that the universe and multiverse repositories are enabled. It should look like this:

#
# deb cdrom:[Ubuntu-Server 9.04 _Jaunty Jackalope_ - Release amd64 (20090421.1)]/ jaunty main restricted

#deb cdrom:[Ubuntu-Server 9.04 _Jaunty Jackalope_ - Release amd64 (20090421.1)]/ jaunty main restricted
# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.

deb http://de.archive.ubuntu.com/ubuntu/ jaunty main restricted
deb-src http://de.archive.ubuntu.com/ubuntu/ jaunty main restricted

## Major bug fix updates produced after the final release of the
## distribution.
deb http://de.archive.ubuntu.com/ubuntu/ jaunty-updates main restricted
deb-src http://de.archive.ubuntu.com/ubuntu/ jaunty-updates main restricted

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team. Also, please note that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
deb http://de.archive.ubuntu.com/ubuntu/ jaunty universe
deb-src http://de.archive.ubuntu.com/ubuntu/ jaunty universe
deb http://de.archive.ubuntu.com/ubuntu/ jaunty-updates universe
deb-src http://de.archive.ubuntu.com/ubuntu/ jaunty-updates universe

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team, and may not be under a free licence. Please satisfy yourself as to
## your rights to use the software. Also, please note that software in
## multiverse WILL NOT receive any review or updates from the Ubuntu
## security team.
deb http://de.archive.ubuntu.com/ubuntu/ jaunty multiverse
deb-src http://de.archive.ubuntu.com/ubuntu/ jaunty multiverse
deb http://de.archive.ubuntu.com/ubuntu/ jaunty-updates multiverse
deb-src http://de.archive.ubuntu.com/ubuntu/ jaunty-updates multiverse

## Uncomment the following two lines to add software from the 'backports'
## repository.
## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
# deb http://de.archive.ubuntu.com/ubuntu/ jaunty-backports main restricted universe multiverse
# deb-src http://de.archive.ubuntu.com/ubuntu/ jaunty-backports main restricted universe multiverse

## Uncomment the following two lines to add software from Canonical's
## 'partner' repository.
## This software is not part of Ubuntu, but is offered by Canonical and the
## respective vendors as a service to Ubuntu users.
# deb http://archive.canonical.com/ubuntu jaunty partner
# deb-src http://archive.canonical.com/ubuntu jaunty partner

deb http://security.ubuntu.com/ubuntu jaunty-security main restricted
deb-src http://security.ubuntu.com/ubuntu jaunty-security main restricted
deb http://security.ubuntu.com/ubuntu jaunty-security universe
deb-src http://security.ubuntu.com/ubuntu jaunty-security universe
deb http://security.ubuntu.com/ubuntu jaunty-security multiverse
deb-src http://security.ubuntu.com/ubuntu jaunty-security multiverse

Then run the following to update the apt package database:

aptitude update

Run the following to install the latest updates:

aptitude safe-upgrade

If you see that a new kernel gets installed as part of the updates, you should reboot the system afterwards:

 

5. Change The Default Shell

/bin/sh is a symlink to /bin/dash, however we need /bin/bash, not /bin/dash. Therefore we do this:

dpkg-reconfigure dash

Install dash as /bin/sh? <– No

 

6. Disable AppArmor

AppArmor is a security extension (similar to SELinux) that should provide extended security. In my opinion you don’t need it to configure a secure system, and it usually causes more problems than advantages (think of it after you have done a week of trouble-shooting because some service wasn’t working as expected, and then you find out that everything was ok, only AppArmor was causing the problem). Therefore I disable it (this is a must if you want to install ISPConfig later on).

We can disable it like this:

/etc/init.d/apparmor stop
update-rc.d -f apparmor remove
aptitude remove apparmor apparmor-utils

 

7. Install Some Software

Now we install a few packages that are needed later on:

aptitude install binutils cpp fetchmail flex gcc libarchive-zip-perl libc6-dev libcompress-zlib-perl libdb4.6-dev libpcre3 libpopt-dev lynx m4 make ncftp nmap openssl perl perl-modules unzip zip zlib1g-dev autoconf automake1.9 libtool bison autotools-dev g++ build-essential unrar

 

8. Synchronize the System Clock

It is a good idea to synchronize the system clock with an NTP (network time protocol) server over the internet. Simply run :

apt-get install ntp ntpdate

 

9. Caching Dnsmasq

apt-get install dnsmasq

Edit /etc/dnsmasq.conf and make Dnsmasq listen on localhost:

listen-address=127.0.0.1

Edit /etc/resolv.conf and add the following to the top of the list:

nameserver 127.0.0.1

10. Apache2

apt-get install apache2 apache2-suexec apache2-doc apache2-mpm-prefork apache2-utils libexpat1

Next we install PHP5 as an Apache module:

aptitude install libapache2-mod-php5 php5 php5-common php5-curl php5-dev php5-gd php5-idn php-pear php5-imagick php5-imap php5-mcrypt php5-memcache php5-mhash php5-ming php5-mysql php5-pspell php5-recode php5-snmp php5-sqlite php5-tidy php5-xmlrpc php5-xsl

Next we edit /etc/apache2/mods-available/dir.conf and change the DirectoryIndex line:

<IfModule mod_dir.c>

          #DirectoryIndex index.html index.cgi index.pl index.php index.xhtml index.htm
          DirectoryIndex index.html index.htm index.shtml index.cgi index.php index.php3 index.pl index.xhtml

</IfModule>

Now we have to enable some Apache modules (rewrite, suexec, include)

a2enmod rewrite
a2enmod suexec
a2enmod include

Restart Apache:

/etc/init.d/apache2 restart

11. Postfix and MySQL

Install the packages:

apt-get install postfix postfix-mysql postfix-doc mysql-server mysql-client procmail

MySQL:

You will be asked to provide a password for the MySQL root user – this password is valid for the user root@localhost as well as root@server1.example.com, so we don’t have to specify a MySQL root password manually later on:

New password for the MySQL “root” user: <– yourrootsqlpassword
Repeat password for the MySQL “root” user: <– yourrootsqlpassword

Postfix:

You will be asked two questions. Answer as follows:

General type of mail configuration: <– Internet Site
System mail name: <– server1.example.com

We want MySQL to listen on all interfaces, not just localhost, therefore we edit /etc/mysql/my.cnf and comment out the line bind-address = 127.0.0.1:

[...]
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
#bind-address           = 127.0.0.1
[...]

Then we restart MySQL:

/etc/init.d/mysql restart

Now check that networking is enabled. Run

netstat -tap | grep mysql

The output should look like this:

tcp        0      0 *:mysql                 *:*                     LISTEN      4318/mysqld

Stop Postfix:

postfix stop

We’ll want to edit Postfix with the below:

Edit master.cf:

We need to add two items below the pickup service type. The pickup service “picks up” local mail (local meaning “on this machine”) and delivers it. This is a way to bypass content filtering for mail generated by this machine.

It should look like this when you are done:

pickup    fifo  n       -       -       60      1       pickup
         -o content_filter=
         -o receive_override_options=no_header_body_checks

Edit main.cf:

postconf -e “alias_maps = hash:/etc/aliases”
newaliases
postconf -e “myorigin = example.com”
postconf -e “myhostname = server1.example.com”
postconf -e “mynetworks = 127.0.0.0/8, 192.168.0.0/24”
postconf -e “message_size_limit = 10485760”
postconf -e “local_transport = error:No local mail delivery”
postconf -e “mydestination = ”
postconf -e “local_recipient_maps = ”
postconf -e “virtual_alias_maps = hash:/etc/postfix/virtual”

Create /etc/postfix/virtual and add the following:

postmaster postmaster@example.com
abuse abuse@example.com
root root@example.com

Continue:

postmap /etc/postfix/virtual
postconf -e “relay_recipient_maps = hash:/etc/postfix/relay_recipients”

Create /etc/postfix/relay_recipients and add the following:

@example.com OK
@example2.com OK

Continue:

postmap /etc/postfix/relay_recipients
postconf -e “transport_maps = hash:/etc/postfix/transport”

Create /etc/postfix/transport and add the following:

example.com smtp:[192.168.0.x]
example2.com smtp:[192.168.0.x]

Continue:

postmap /etc/postfix/transport
postconf -e “relay_domains = hash:/etc/postfix/relay_domains”

Create /etc/postfix/relay_domains and add the following:

example.com OK
example2.com OK

Continue:

postmap /etc/postfix/relay_domains
postconf -e “smtpd_helo_required = yes”
postconf -e “smtpd_sender_restrictions = reject_non_fqdn_sender, reject_unknown_sender_domain, permit”
postconf -e “smtpd_recipient_restrictions = reject_non_fqdn_sender, reject_non_fqdn_recipient, reject_unknown_recipient_domain, permit_mynetworks, reject_unauth_destination, reject_unauth_pipelining, reject_rbl_client zen.spamhaus.org, reject_rbl_client bl.spamcop.net, permit”
postconf -e “smtpd_data_restrictions = reject_unauth_pipelining”

Let’s take a final look at the Postfix configuration:

less /etc/postfix/main.cf

Check the contents of the file for errors and repair if needed. Fire up Postfix:

postfix start

Check that Postfix responds:

telnet 127.0.0.1 25

You should see:

220 [yourFQDNhere] ESMTP Postfix (Ubuntu)

12. MailScanner Razor Pyzor DCC Clamav Installation

apt-get install mailscanner razor pyzor clamav-daemon

Let’s start with MailScanner. The MailScanner that was just installed from the repositories is a very old version so we will now remove it and install the MailScanner package from source. The above is done to install the dependencies only.

apt-get remove mailscanner

Download http://www.mailscanner.info/files/4/tar/MailScanner-install-4.75.11-1.tar.gz into /usr/src/mailscanner/ and run:

tar xvfz MailScanner-install-4.75.11-1.tar.gz
cd MailScanner-install-4.75.11
./install.sh

Disable the default MailScanner:

mv /etc/MailScanner /etc/MailScanner.dist

13. Pyzor Configuration

Because pyzor doesn’t work with python2.6 very well, the workaround is to install python2.5 alongside it and use it as the default.

Edit /usr/share/python/debian_defaults:

# the default python version
default-version = python2.5

Now modify the /usr/bin/python symlink to point to python2.5 instead to python2.6

mv /usr/bin/python /usr/bin/python26
ln -s /usr/bin/python2.5 /usr/bin/python

Move pyzor:

mv /var/lib/python-support/python2.6/pyzor-* /var/lib/python-support/python2.5/
mv /var/lib/python-support/python2.6/pyzor* /var/lib/python-support/python2.5/pyzor/

We need to change some permissions on pyzor first:

chmod -R a+rX /usr/share/doc/pyzor /usr/bin/pyzor /usr/bin/pyzord
chmod -R a+rX /usr/share/python-support/pyzor

Here we supply the IP address of the Pyzor server to Pyzor. This will create the server’s IP address in a servers file therein. Then it will test the connection. If you are behind a firewall, open port 24441/udp in and out to your server. While you’re at it also open up 6277/udp for DCC, 2703/tcp for Razor and 783/tcp for SpamAssassin:

pyzor –homedir /var/lib/MailScanner discover
pyzor ping

 

14. Razor Configuration

Create the .razor configuration:

cd
rm /etc/razor/razor-agent.conf
mkdir /var/lib/MailScanner/.razor
razor-admin -home=/var/lib/MailScanner/.razor -create
razor-admin -home=/var/lib/MailScanner/.razor -discover
razor-admin -home=/var/lib/MailScanner/.razor -register
chown -R postfix:www-data /var/lib/MailScanner
chmod -R ug+rwx /var/lib/MailScanner

Make the following changes to /var/lib/MailScanner/.razor/razor-agent.conf:

debuglevel = 0
razorhome = /var/lib/MailScanner/.razor/

 

15. DCC Configuration

Install DCC from .deb source:

wget http://ppa.launchpad.net/jonasped/ppa/ubuntu/pool/main/d/dcc/dcc-server_1.3.103-0ubuntu1~ppa2~jaunty1_i386.deb
wget http://ppa.launchpad.net/jonasped/ppa/ubuntu/pool/main/d/dcc/dcc-common_1.3.103-0ubuntu1~ppa2~jaunty1_i386.deb
wget http://ppa.launchpad.net/jonasped/ppa/ubuntu/pool/main/d/dcc/dcc-client_1.3.103-0ubuntu1~ppa2~jaunty1_i386.deb

Install the packages:

dpkg -i dcc-common_1.3.103-0ubuntu1~ppa2~jaunty1_i386.deb
dpkg -i dcc-server_1.3.103-0ubuntu1~ppa2~jaunty1_i386.deb
dpkg -I dcc-client_1.3.103-0ubuntu1~ppa2~jaunty1_i386.deb

We are not running a DCC server, so we don’t need to waste time checking ourselves.
Once the installation is done run:

cdcc “delete 127.0.0.1”
cdcc “delete 127.0.0.1 Greylist”

Test our installation with:

cdcc info

You should get ‘requests ok’ from the servers.

 

16. Configuring MailScanner and ClamAV

We need to make a directory for SpamAssassin in the spool and give postfix permissions to it, if you run sa-learn –force as root, bayes databese that is stored in these directories will change to root:root and spamassassin will error looking at the db. Just keep an eye on the mail.log and you’ll remember to change the permissions back. Also disable the MailScanner default configs:

mkdir /var/spool/MailScanner/spamassassin

Backup your MailScanner.conf file:

cp /opt/MailScanner/etc/MailScanner.conf /opt/MailScanner/etc/MailScanner.conf.dist

vi /opt/MailScanner/etc/MailScanner.conf

Change the following parameters in MailScanner.conf:

%org-name% = ORGNAME
%org-long-name% = ORGFULLNAME
%web-site% = ORGWEBSITE
Run As User = postfix
Run As Group = www-data
Incoming Work Group = clamav
Incoming Work Permissions = 0640
Incoming Queue Dir = /var/spool/postfix/hold
Outgoing Queue Dir = /var/spool/postfix/incoming
MTA = postfix
Virus Scanners = clamd
Spam Subject Text = ***SPAM***
Send Notices = no
Spam List = spamcop.net ZEN
Required SpamAssassin Score = 6
High SpamAssassin Score = 10
Spam Actions = deliver
High Scoring Spam Actions = delete
Rebuild Bayes Every = 0
Wait During Bayes Rebuild = no
SpamAssassin User State Dir = /var/spool/MailScanner/spamassassin
Clamd Socket = /var/run/clamav/clamd.ctl
Clamd Lock File = /var/run/clamav/clamd.pid
Monitors for ClamAV Updates = /var/lib/clamav/*.cld /var/lib/clamav/*.cvd

The first 9 lines are basically required in order for everything to work, the rest are recommended.

16.1 header_checks

Let’s go ahead and put this in main.cf. header_checks is required because it allows us to hold all incoming email in order for MailScanner to do its thing:

postconf -e “header_checks = regexp:/etc/postfix/header_checks”

Create /etc/postfix/header_checks and add the following:

/^Received:/ HOLD

Postmap it:

postmap /etc/postfix/header_checks

 

16.2 Fix to Disable Permission Checks on MailScanner Directories

Edit /etc/rc2.d/S20mailscanner to look like:

check_dir /var/spool/MailScanner       ${user:-postfix} ${group:-www-data}
#check_dir /var/lib/MailScanner         ${user:-mail} ${group:-mail}
#check_dir /var/run/MailScanner         ${user:-mail} ${group:-mail}
#check_dir /var/lock/subsys/MailScanner       ${user:-postfix} ${group:-www-data}

In the file /etc/default/mailscanner, make sure this parameter is at 1:

run_mailscanner=1

 

16.3 Create cron jobs to keep MailScanner updated:

37      5 * * * /opt/MailScanner/bin/update_phishing_sites
07      * * * * /opt/MailScanner/bin/update_bad_phishing_sites
42      * * * * /opt/MailScanner/bin/update_virus_scanners
3,23,43 * * * * /opt/MailScanner/bin/check_mailscanner

 

16.4 Start the system

check_mailscanner
/etc/init.d/postfix start

Check your logs for errors:

tail -f /var/log/mail.log

17. MailWatch Installation Instructions

This setup assumes you are using Apache v2.x and not Apache v1.x.

Before Starting, make sure that MailScanner is working before you continue with the MailWatch install!

*Notes for Ubuntu:

You must have a working MailScanner set-up and running copies of MySQL, Apache, and PHP. You must also have the Perl DBD-MySQL package installed for the Perl portions of MailScanner to utilize the MySQL database.

The default php.ini set should have the following set correctly, you may want to check this:

short_open_tag = On
safe_mode = Off
register_globals = Off
magic_quotes_gpc = On
magic_quotes_runtime = Off
session.auto_start = 0

These will be commented out you must remove the “#” to activate them:

extension=mysql.so
extension=gd.so

 

17.1 Installation

Download the latest MailWatch release:

wget http://downloads.sourceforge.net/mailwatch/mailwatch-1.0.4.tar.gz?modtime=1178902008&big_mirror=0
tar xzvf mailwatch-1.0.4.tar.gz
cd mailwatch-1.0.4

17.2 Create the database

mysql -p < create.sql

*Note: you will need to modify the above as necessary for your system if you have a root password for your MySQL database (recommended!) – Debian will ask for one.

Create a MySQL user and password & Set-up MailScanner for SQL logging:

mysql -p
mysql> GRANT ALL ON mailscanner.* TO mailwatch@localhost IDENTIFIED BY ‘password’;

Remember the password! You need the single quotes ‘ to surround your password.

17.3 Edit and move MailWatch.pm

Edit MailWatch.pm and change the $db_user and $db_pass values accordingly and move MailWatch.pm:

mv MailWatch.pm /opt/MailScanner/lib/MailScanner/CustomFunctions/

 

17.4 Create a MailWatch Web User

mysql mailscanner -u mailwatch -p

Enter password: ******

mysql> INSERT INTO users VALUES (‘username’,md5(‘password’),’mailscanner’,’A’,’0′,’0′,’0′,’0′,’0′);

17.5 Install & Configure MailWatch

From within the unpacked mailwatch directory move the directory called ‘mailscanner’ to /var/www/.

mv mailscanner/ /var/www/
cd /var/www/mailscanner

Make a temp directory:

mkdir temp
chgrp www-data temp
chmod g+w temp

Check the permissions of /var/www/mailscanner/images and /var/www/images/cache – they should be ug+rwx and owned by root and in the same group as the web server user.

chown root:www-data images
chmod ug+rwx images
chown root:www-data images/cache
chmod ug+rwx images/cache

Create conf.php by copying conf.php.example and edit the values to suit, you will need to set DB_USER and DB_PASS to the MySQL user and password that you created earlier.

Change these values as shown below:

define(DB_USER, 'mailwatch');
define(DB_PASS, 'password');
define(MAILWATCH_HOME, '/var/www/mailscanner');
define(MS_LIB_DIR, '/usr/lib/MailScanner/');
define(QUARANTINE_USE_FLAG, true);
define(QUARANTINE_FROM_ADDR, 'postmaster@example.com');
define(QUARANTINE_USE_SENDMAIL, true);

17.6 Set-up MailScanner

Edit /opt/MailScanner/etc/MailScanner.conf and make sure the following are set:

Quarantine User = root
Quarantine Group = www-data
Quarantine Permissions = 0640
Quarantine Whole Message = yes
Always Looked Up Last = &MailWatchLogging
Quarantine Whole Message As Queue Files = no
Detailed Spam Report = yes
Include Scores In SpamAssassin Report = yes

*Important: Spam Actions, High Scoring Spam Actions and No Spam Actions should also have ‘store’ as one of the keywords if you want to quarantine those items for bayes learning or viewing from within MailWatch.

17.7 Integrate SQL Blacklist/Whitelist (optional)

If you would like to manage the MailScanner whitelist and blacklist from within the MailWatch web interface perform the following steps.

1. Edit the MySQL connection values within the CreateList subroutine of SQLBlackWhiteList.pm to match the values you entered previously into MailWatch.pm. Both files should contain the same values. (Look for the following lines in SQLBlackWhiteList.pm and enter your own data.)

my($db_user) = 'mailwatch';
my($db_pass) = 'password';

2. Copy SQLBlackWhiteList.pm to /opt/MailScanner/lib/MailScanner/CustomFunctions/.

3. Edit MailScanner.conf and set:

Is Definitely Not Spam = &SQLWhitelist
Is Definitely Spam = &SQLBlacklist

17.8 Fix to allow MailWatch to work with Postfix Inbound/Outbound Queue

Download the patch from http://www.gbnetwork.co.uk/mailscanner/files/postfixmail.tar.gz

cd /usr/src
wget http://www.gbnetwork.co.uk/mailscanner/files/postfixmail.tar.gz
tar xvfz postfixmail.tar.gz
cd postfixmail
cp postfix* /var/www/mailscanner/
patch /var/www/mailscanner/functions.php functions.php.diff

 

17.9 SpamAssassin

First we need to disable the default SpamAssassin configuration file:

mv /etc/spamassassin/local.cf /etc/spamassassin/local.cf.disabled

Now let’s backup the SpamAssassin configuration file in MailScanner then edit:

cp /opt/MailScanner/etc/spam.assassin.prefs.conf /opt/MailScanner/etc/spam.assassin.prefs.conf.back

Add pyzor and razor paths to the top of /opt/MailScanner/etc/spam.assassin.prefs.conf:

pyzor_options –homedir /var/lib/MailScanner/
razor_config /var/lib/MailScanner/.razor/razor-agent.conf

Look for these lines and change them accordingly:

bayes_ignore_header X-YOURDOMAIN-COM-MailScanner
bayes_ignore_header X-YOURDOMAIN-COM-MailScanner-SpamCheck
bayes_ignore_header X-YOURDOMAIN-COM-MailScanner-SpamScore
bayes_ignore_header X-YOURDOMAIN-COM-MailScanner-Information

“YOURDOMAIN-COM” should be replaced with whatever you used for “%org-name%” in the MailScanner.conf file. Leave the “X-” in place. This is the same orgname used in the MailScanner.conf above.

Edit the SpamAssassin v310.pre to enable Razor and DCC:

loadplugin Mail::SpamAssassin::Plugin::DCC
loadplugin Mail::SpamAssassin::Plugin::Razor2

17.10 SpamAssassin SQL Bayes

Pre-requisites

a. You’ll need the perl-DBI and perl-DBD-MySQL modules installed.

Assumptions and Variables:

SpamAssassin Bayes Database Name: sa_bayes
SpamAssassin Bayes Database UserName: sa_user
SpamAssassin Bayes Database Password: sa_password

Create the MySQL database:

First of all, create a database on the server where you intend on storing the bayesian information.

mysql -u root -p

mysql> create database sa_bayes;
mysql> GRANT ALL ON sa_bayes.* TO sa_user@localhost IDENTIFIED BY ‘sa_password’;
mysql> flush privileges;

Locate the bayes_mysql.sql file:

find / -name bayes_mysql.sql
mysql -u sa_user -p sa_bayes < /path/to/bayes_mysql.sql

Make some changes to your spam.assassin.prefs.conf:

bayes_store_module Mail::SpamAssassin::BayesStore::SQL
bayes_sql_dsn DBI:mysql:sa_bayes:localhost
bayes_sql_username sa_user
bayes_sql_password sa_password
bayes_sql_override_username root

Make sure the following is commented:

#bayes_path /etc/MailScanner/bayes/bayes
#bayes_file_mode 0660

Also add this to your crontab:

30 01 * * * /usr/bin/sa-learn --force-expire --sync -p /opt/MailScanner/etc/spam.assassin.prefs.conf 

17.11 Set permissions to bring it all together

chown -R postfix:www-data /var/spool/MailScanner
chown -R postfix:www-data /var/lib/MailScanner
chown -R postfix:www-data /var/spool/postfix/hold
chmod -R ug+rwx /var/spool/postfix/hold
chmod -R u+rwx,g+rx /var/spool/MailScanner/quarantine

Finally make sure you restart MailScanner:

killall mailscanner

check_mailscanner

Test out the setup:

spamassassin -x -D -p /opt/MailScanner/etc/spam.assassin.prefs.conf –lint

Check for lines like:

debug: bayes: Database connection established
debug: bayes: found bayes db version 3
debug: bayes: Using userid: 2

You should see lines come up with DCC, Pyzor and Razor that say loading plugin and hopefully no errors.

Finishing up this part we need to add cron jobs that will clean/update, you probably saw the message about this after the MailScanner install script finished.

First edit conf.php and set ‘QUARANTINE_DAYS_TO_KEEP’ in conf.php and change the following line in /usr/src/mailwatch-1.0.4/tools/db_clean to:

#!/usr/bin/php -q
require('/var/www/mailscanner/functions.php');

Install quarantine clean up script:

cp /usr/src/mailwatch-1.0.4/tools/quarantine_maint.php /usr/bin/quarantine_maint.php
cp /usr/src/mailwatch-1.0.4/tools/db_clean.php /usr/bin/db_clean.php
chmod +x /usr/bin/quarantine_maint.php
chmod +x /usr/bin/db_clean.php

Run

crontab -e

and add the following:

15 10 * * 2 /usr/bin/quarantine_maint.php --clean &> /dev/null
58 23 * * * /usr/bin/db_clean.php &> /dev/null

Disable the mailscanner installed cron script /etc/cron.daily/clean.quarantine (note: do this only if the clean.quarantine script exists).

$disabled = 1;

17.12 Reboot the system

reboot

Check your mail.log (tail –f /var/log/mail.log) and you should see the following:

Jun 13 12:18:23 hoshi MailScanner[26388]: MailScanner E-Mail Virus Scanner version 4.20-3 starting…
Jun 13 12:18:24 hoshi MailScanner[26388]: Config: calling custom init function MailWatchLogging
Jun 13 12:18:24 hoshi MailScanner[26388]: Initialising database connection
Jun 13 12:18:24 hoshi MailScanner[26388]: Finished initialising database connection

Congratulations – you now have MailScanner logging to MySQL.

17.13 Test the MailWatch interface

Point your browser to http://<hostname>/mailscanner/ – you should be prompted for a username and password – enter the details of the MailWatch web user that you created earlier, and you should see a list of the last 50 messages processed by MailScanner.

If you’re not able to see the mails, then you may have to set the following persmissions:

chgrp -R www-data /var/spool/MailScanner

You may have to create the following to prevent an error in a lint test:

mkdir /var/www/.spamassassin

17.14 Update the SpamAssassin Rules table

MailWatch keeps a list of all the SpamAssassin rules and descriptions which are displayed on the ‘Message Detail’ page – to show the descriptions, you need to run the updater every time you add new rules or upgrade SpamAssassin. Click on the ‘Tools/Links’ menu and select ‘Update SpamAssassin Rule Descriptions’ and click ‘Run Now’.

17.15 Update the GeoIP database

Change this line in /var/www/mailscanner/geoip_update.php to look like:

dbquery("LOAD DATA LOCAL INFILE

*Note: Make sure that allow_url_fopen = On is set in your php.ini.

Click on the ‘Tools/Links’ menu and select ‘Update GeoIP database’ and click ‘Run Now’.

17.16 Fix to allow wildcards in Whitelist/Blacklist

Add the following to the bottom of the return 1 section in your /opt/MailScanner/lib/MailScanner/CustomFunctions/SQLBlackWhiteList.pm:

return 1 if $BlackWhite->{$to}{'*@'.$fromdomain};
return 1 if $BlackWhite->{$to}{'*@*.'.$fromdomain};
return 1 if $BlackWhite->{$todomain}{'*@'.$fromdomain};
return 1 if $BlackWhite->{$todomain}{'*@*.'.$fromdomain};
return 1 if $BlackWhite->{'default'}{'*@'.$fromdomain};
return 1 if $BlackWhite->{'default'}{'*@*.'.$fromdomain};

17.17 Fix for Message Operations Not Finding Messages

Change the following in /var/www/mailscanner/do_message_ops.php file:

$id = $Regs[1];

to

$id = str_replace("_", ".",$Regs[1]);

17.18 Releasing Spam Messages

To allow MailWatch to release Spam messages without them being processed again, add 127.0.0.1 as a whitelist item in MailWatch/List interface.  Make sure to restart MailScanner after configuring these options.  Below is what my entry looks like.

127.0.0.1 default Delete

17.19 Fix to Allow Multiple Releases of Messages in Message Operations

Edit /var/www/mailscanner/do_message_ops.php and make the following changes:

   case 'F':
    $type='forget';
    break;
   case 'R':
    $type='release';
    break;
   default:
    continue;
    break;

Then, find the following section and change it to look like this:

$itemnum = array($num);
   if ($type == 'release'){
      if($quarantined = quarantine_list_items($id,RPC_ONLY)) {
         $to = $quarantined[0]['to'];
       }
       echo "<tr><td><a href=\"detail.php?id=$id\">$id</a></td><td>$type</td><td>" . quarantine_release($quarantined, $itemnum, $to, RPC_ONLY) . "</td></tr>\n";
    } else {
     echo "<tr><td><a href=\"detail.php?id=$id\">$id</a></td><td>$type</td><td>" . quarantine_learn($items, $itemnum, $type, RPC_ONLY) . "</td></tr>\n";
     }
   }
  echo "</TABLE>\n";
      }
    }
  }
}
echo "  </TD>\n";

Next we edit the /var/www/mailscanner/functions.php file and change:

$fieldname[$f] = "Ops<br>S&nbsp;&nbsp;H&nbsp;&nbsp;F";

to

$fieldname[$f] = "Ops<br>S&nbsp;&nbsp;H&nbsp;&nbsp;F&nbsp;&nbsp;R";

Next change:

array_unshift($row, "<INPUT NAME=\"OPT-REPLACEME\" TYPE=RADIO VALUE=\"S\">&nbsp;<INPUT NAME=\"OPT-REPLACEME\" TYPE=RADIO VALUE=\"H\">&nbsp;<INPUT NAME=\"OPT-REPLACEME\" TYPE=RADIO VALUE=\"F\">");

To:

array_unshift($row, "<INPUT NAME=\"OPT-REPLACEME\" TYPE=RADIO VALUE=\"S\">&nbsp;<INPUT NAME=\"OPT-REPLACEME\" TYPE=RADIO VALUE=\"H\">&nbsp;<INPUT NAME=\"OPT-REPLACEME\" TYPE=RADIO VALUE=\"F\">&nbsp;<INPUT NAME=\"OPT-REPLACEME\" TYPE=RADIO VALUE=\"R\"> ");

Next find the block with the javascript function to handle radio buttons. Add a third value like so:

echo "function SetRadios(p) {\n";
echo " var val;\n";
echo " if (p == 'S') {\n";
echo "  val = 0;\n";
echo " } else if (p == 'H') {\n";
echo "  val = 1;\n";
echo " } else if (p == 'F') {\n";
echo "  val = 2;\n";
echo " } else if (p == 'R') {\n";
echo "  val = 3;\n";
echo " } else if (p == 'C') {\n";
echo "  ClearRadios();\n";

Now, add the text for the radios:

echo "&nbsp; <a href=\"javascript:SetRadios('S')\">S</a>";
echo "&nbsp; <a href=\"javascript:SetRadios('H')\">H</a>";
echo "&nbsp; <a href=\"javascript:SetRadios('F')\">F</a>";
echo "&nbsp; <a href=\"javascript:SetRadios('R')\">R</a>";

Finally, change:

echo "<P><b>S</b> = Spam &nbsp; <b>H</b> = Ham &nbsp; <b>F</b> = Forget\n";

to:

echo "<P><b>S</b> = Spam &nbsp; <b>H</b> = Ham &nbsp; <b>F</b> = Forget &nbsp; <b>R</b> = Release\n";

17.20 Patch to fix autocommit error when stopping MailScanner

Edit line 80 of MailWatch.pm to look like this:

$dbh = DBI->connect("DBI:mysql:database=$db_name;host=$db_host", $db_user, $db_pass, {PrintError => 0, AutoCommit => 0});

*Note: This turns off autocommit when it connects so the commit line when it closes does not throw an error.

18. Install and Configure SPF

The postfix-policyd-spf-perl package depends on the Mail::SPF and the NetAddr::IP Perl modules.

We need to download postfix-policyd-spf-perl from http://www.openspf.org/Software to the /usr/src/ directory and install it to the /usr/lib/postfix/ directory like this:

cd /usr/src
wget http://www.openspf.org/blobs/postfix-policyd-spf-perl-2.005.tar.gz
tar xvfz postfix-policyd-spf-perl-2.005.tar.gz
cd postfix-policyd-spf-perl-2.005
cp postfix-policyd-spf-perl /usr/lib/postfix/policyd-spf-perl

Then we edit /etc/postfix/master.cf and add the following stanza at the end:

policy unix - n n - - spawn
   user=nobody argv=/usr/bin/perl /usr/lib/postfix/policyd-spf-perl

(The leading spaces before user=nobody are important so that Postfix knows that this line belongs to the previous one!)

Then open /etc/postfix/main.cf and search for the smtpd_recipient_restrictions directive. You should have reject_unauth_destination in that directive, and right after reject_unauth_destination you add check_policy_service unix:private/policy like this:

[...]
smtpd_recipient_restrictions =
[...]
   reject_unauth_destination
   check_policy_service unix:private/policy
[...]

It is important that you specify check_policy_service AFTER reject_unauth_destination or else your system can become an open relay!

Then restart Postfix:

/etc/init.d/postfix restart

That’s it already.

 

 

Comments

comments