Cheap VPS & Xen Server

Residential Proxy Network - Hourly & Monthly Packages

The Perfect SpamSnake – Ubuntu Jeos 9.10


Postfix w/Bayesian Filtering and Anti-Backscatter (Relay Recipients), Apache, Mysql, Dnsmasq, MailScanner (Spamassassin, ClamAV, Pyzor, Razor, DCC-Client), MailWatch, SPF Checks, FuzzyOcr, Sanesecurity Signatures, SQLGrey, KAM, Scamnailer, FireHOL (Iptables Firewall), Relay Recipients, Webmin (Optional), Outgoing Disclaimer with alterMIME (Optional)

This tutorial shows how to set up an Ubuntu Jeos based server as a spamfilter in Gateway mode. In the end, you will have a SpamSnake Gateway which will relay clean emails to your MTA. You will also be able to view your incoming queue, train your SpamSnake and carry out a few more advanced operations via MailWatch.

I cannot offer any guarantees that this will work for you, the same way it’s working for me.

I will use the following software:
• Web Server: Apache 2 with PHP 5
• Database Server: MySQL 5.1.37
• Mail Server: Postfix v2.6.5
• Caching DNS Server: Dnsmasq v2.50
• MailScanner: MailScanner v4.78
• MailWatch: MailWatch v1.0.4

Credit goes to the guys at Kreationnext and the developers of MailScanner, MailWatch, Clamav, Apache, Mysql and Postfix.

 

BASE INSTALL

1. Install minimum vm option
Set hostname to server1
2. Default guided partition method
3. Setup user:
u: administrator
p: password
No encryption
4. No auto-updates
5. Install OpenSSH

 

POST INSTALLATION

1. Get root Privileges

Enable the root login by running the following and giving root a password. You can then directly log in as root:

sudo passwd root

Then run the following to update the apt package database:

aptitude update

Run the following to install the latest updates:

aptitude safe-upgrade

If you see that a new kernel gets installed as part of the updates, you should reboot the system afterwards.

 

2. Configure The Network

Because the Ubuntu installer has configured our system to get its network settings via DHCP, we have to change that now because a server should have a static IP address. Edit /etc/network/interfaces and adjust it to your needs (in this example setup I will use the IP address 192.168.0.100):

vi /etc/network/interfaces

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
auto eth0
iface eth0 inet static
        address 192.168.0.100
        netmask 255.255.255.0
        network 192.168.0.0
        broadcast 192.168.0.255
        gateway 192.168.0.1

Then restart your network:

/etc/init.d/networking restart

Then edit /etc/hosts. Make it look like this:

127.0.0.1       localhost.localdomain   localhost
192.168.0.100   server1.example.com     server1
# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

Now run:

echo server1.example.com > /etc/hostname
reboot now

Afterwards, run:

hostname
hostname -f

Both should show server1.example.com now.

 

3. Change The Default Shell

/bin/sh is a symlink to /bin/dash, however we need /bin/bash, not /bin/dash. Therefore we do this:

dpkg-reconfigure dash

Install dash as /bin/sh? <– No

Install a few packages and requirements that are needed later on:

aptitude install binutils cpp fetchmail flex gcc libarchive-zip-perl libc6-dev libcompress-zlib-perl libdb4.6-dev libpcre3 libpopt-dev lynx m4 make ncftp nmap openssl perl perl-modules unzip zip zlib1g-dev autoconf automake1.9 libtool bison autotools-dev g++ build-essential telnet wget gawk

4. Caching Dnsmasq

apt-get install dnsmasq

Edit /etc/dnsmasq.conf and make Dnsmasq listen on localhost:

listen-address=127.0.0.1

Edit /etc/resolv.conf and append the following to the top of the list:

nameserver 127.0.0.1

 

5. Install Apache2 and Mysql

apt-get install apache2 php5-cli libapache2-mod-php5 mysql-client mysql-server libdbd-mysql-perl php5-gd php5-mysql

 

Install MySql

You will be asked to provide a password for the MySQL root user – this password is valid for the user root@localhost as well as root@server1.example.com, so we don’t have to specify a MySQL root password manually later on:

New password for the MySQL “root” user: <– yourrootsqlpassword
Repeat password for the MySQL “root” user: <– yourrootsqlpassword

 

Install Apache2

apt-get install apache2 apache2-doc apache2-mpm-prefork apache2-utils apache2-suexec

Next we install PHP5 as an Apache module:

aptitude install libapache2-mod-php5 php5 php5-common php5-curl php5-dev php5-gd php5-idn php-pear php5-imagick php5-imap php5-mcrypt php5-memcache php5-mhash php5-ming php5-mysql php5-pspell php5-recode php5-snmp php5-sqlite php5-tidy php5-xmlrpc php5-xsl

Next we edit /etc/apache2/mods-available/dir.conf and change the DirectoryIndex line:

<IfModule mod_dir.c>
          #DirectoryIndex index.html index.cgi index.pl index.php index.xhtml index.htm
          DirectoryIndex index.html index.htm index.shtml index.cgi index.php index.php3 index.pl index.xhtml
</IfModule>

Now we have to enable some Apache modules (rewrite, suexec, include):

a2enmod rewrite
a2enmod suexec
a2enmod include

Restart Apache:

/etc/init.d/apache2 restart

 

Install Postfix:

Install the packages:

apt-get install postfix postfix-mysql postfix-doc procmail

 

Postfix:

You will be asked two questions. Answer as follows:

General type of mail configuration: –> Internet Site
System mail name: –> server1.example.com

Stop Postfix:

postfix stop

We’ll want to edit Postfix with the below:

Edit master.cf:
We need to add two items below the pickup service type. The pickup service “picks up” local mail (local meaning “on this machine”) and delivers it. This is a way to bypass content filtering for mail generated by this machine.

It should look like this when you are done:

pickup    fifo  n       -       -       60      1       pickup
         -o content_filter=
         -o receive_override_options=no_header_body_checks

Note: For this step, make sure to replace anything@example.com, example.com and @example.com with real values that matches your setup.

Edit main.cf:

postconf -e “alias_maps = hash:/etc/aliases”
newaliases
postconf -e “myorigin = example.com”
postconf -e “myhostname = server1.example.com”
postconf -e “mynetworks = 127.0.0.0/8, 192.168.0.0/24”
postconf -e “message_size_limit = 10485760”
postconf -e “local_transport = error:No local mail delivery”
postconf -e “mydestination = ”
postconf -e “local_recipient_maps = ”
postconf -e “virtual_alias_maps = hash:/etc/postfix/virtual”

Create /etc/postfix/virtual and add the following:

postmaster postmaster@example.com
abuse abuse@example.com
root root@example.com

Continue:

postmap /etc/postfix/virtual
postconf -e “relay_recipient_maps = hash:/etc/postfix/relay_recipients”

Create /etc/postfix/relay_recipients and add the following:

@example.com OK
@example2.com OK

Continue:

postmap /etc/postfix/relay_recipients
postconf -e “transport_maps = hash:/etc/postfix/transport”

Create /etc/postfix/transport and add the following:

example.com smtp:[192.168.0.x]
example2.com smtp:[192.168.0.x]

Continue:

postmap /etc/postfix/transport
postconf -e “relay_domains = hash:/etc/postfix/relay_domains”

Create /etc/postfix/relay_domains and add the following:

example.com OK
example2.com OK

Continue:

postmap /etc/postfix/relay_domains
postconf -e “smtpd_helo_required = yes”
postconf -e “smtpd_sender_restrictions = reject_non_fqdn_sender, reject_unknown_sender_domain, permit”
postconf -e “smtpd_recipient_restrictions = reject_non_fqdn_recipient, permit_mynetworks, reject_unauth_destination, reject_unauth_pipelining, reject_rbl_client zen.spamhaus.org, reject_rbl_client bl.spamcop.net, permit”
postconf -e “smtpd_data_restrictions = reject_unauth_pipelining”

Final look at the Postfix install:

less /etc/postfix/main.cf

Check the contents of the file for errors and repair if needed. Fire up Postfix:

postfix start

Check that Postfix responds:

telnet 127.0.0.1 25

You should see:

220 [yourFQDNhere] ESMTP Postfix (Ubuntu)

 

6. Install Razor, Pyzor, DCC and Clamav

apt-get install razor pyzor clamav-daemon

 

DCC Configuration

Install DCC from .deb source:

cd /tmp
wget http://ppa.launchpad.net/jonasped/ppa/ubuntu/pool/main/d/dcc/dcc-common_1.3.113-0ubuntu1~ppa1~karmic1_i386.deb && dpkg -i dcc-common_1.3.113-0ubuntu1~ppa1~karmic1_i386.deb
wget http://ppa.launchpad.net/jonasped/ppa/ubuntu/pool/main/d/dcc/dcc-server_1.3.113-0ubuntu1~ppa1~karmic1_i386.deb && dpkg -i dcc-server_1.3.113-0ubuntu1~ppa1~karmic1_i386.deb
wget http://ppa.launchpad.net/jonasped/ppa/ubuntu/pool/main/d/dcc/dcc-client_1.3.113-0ubuntu1~ppa1~karmic1_i386.deb && dpkg -i dcc-client_1.3.113-0ubuntu1~ppa1~karmic1_i386.deb

Test our installation with:

cdcc info

You should get ‘requests ok’ from the servers. Install dependencies:

aptitude install libconvert-tnef-perl libdbd-sqlite3-perl libfilesys-df-perl libmailtools-perl libmime-tools-perl libmime-perl libnet-cidr-perl libsys-syslog-perl libio-stringy-perl libfile-temp-perl libole-storage-lite-perl libarchive-zip-perl libsys-hostname-long-perl libnet-cidr-lite-perl libhtml-parser-perl libdb-file-lock-perl libnet-dns-perl libncurses5-dev libdigest-hmac-perl libdigest-sha1-perl libnet-ip-perl liburi-perl libfile-spec-perl spamassassin libnet-ident-perl libmail-spf-query-perl libmail-dkim-perl dnsutils

Download and install the latest MailScanner:

wget http://www.mailscanner.info/files/4/tar/MailScanner-install-4.78.17-1.tar.gz
tar xvfz MailScanner-install-4.78.17-1.tar.gz && cd MailScanner-install-4.78.17
./install.sh

 

Configuring Pyzor, Razor, MailScanner

Pyzor Configuration

Because pyzor doesn’t work with python2.6 very well, the workaround is to append the following to the first line of /usr/bin/pyzor to make it look like:

#!/usr/bin/python -Wignore::DeprecationWarning

Here we supply the IP address of the Pyzor server to Pyzor. This will create the server’s IP address in a servers file therein. Then it will test the connection. If you are behind a firewall, open port 24441/udp in and out to your server. While you’re at it also open up 6277/udp for DCC, 2703/tcp for Razor and 783/tcp for SpamAssassin:

mkdir /var/lib/MailScanner
pyzor –homedir=/var/lib/MailScanner discover
pyzor ping

 

Razor Configuration

Create the .razor configuration:

cd && rm /etc/razor/razor-agent.conf
mkdir /var/lib/MailScanner/.razor
razor-admin -home=/var/lib/MailScanner/.razor -create
razor-admin -home=/var/lib/MailScanner/.razor -discover
razor-admin -home=/var/lib/MailScanner/.razor -register
chown -R postfix:www-data /var/lib/MailScanner
chmod -R ug+rwx /var/lib/MailScanner

Make the following changes to /var/lib/MailScanner/.razor/razor-agent.conf:

debuglevel        = 0
razorhome 	  = /var/lib/MailScanner/.razor/

 

MailScanner Configuration

We need to make a directory for SpamAssassin in the spool and give postfix permissions to it, if you run sa-learn –force as root, bayes databese that is stored in these directories will change to root:root and spamassassin will error looking at the db. Just keep an eye on the mail.log and you’ll remember to change the permissions back. Also disable the MailScanner default configs:

mkdir /var/spool/MailScanner/spamassassin

Backup your MailScanner.conf file:

cp /opt/MailScanner/etc/MailScanner.conf /opt/MailScanner/etc/MailScanner.conf.dist
vi /opt/MailScanner/etc/MailScanner.conf

Change the following parameters in MailScanner.conf:

%org-name% = ORGNAME
%org-long-name% = ORGFULLNAME
%web-site% = ORGWEBSITE
Run As User = postfix
Run As Group = www-data
Incoming Work Group = clamav
Incoming Work Permissions = 0644
Incoming Queue Dir = /var/spool/postfix/hold
Outgoing Queue Dir = /var/spool/postfix/incoming
MTA = postfix
Virus Scanners = clamd
Monitors for ClamAV Updates = /var/lib/clamav/*.cld /var/lib/clamav/*.cvd
Clamd Socket = /var/run/clamav/clamd.ctl
Clamd Lock File = /var/run/clamav/clamd.pid
Spam Subject Text = ***SPAM***
Send Notices = no
Spam List = spamcop.net ZEN
Spam Actions = deliver store header “X-Spam-Status: Yes”
High Scoring Spam Actions = store delete
Non Spam Actions = deliver store header “X-Spam-Status: No”
SpamAssassin User State Dir = /var/spool/MailScanner/spamassassin

 

header_checks

Let’s go ahead and put this in main.cf. header_checks is required because it allows us to hold all incoming email in order for MailScanner to do its thing:

postconf -e “header_checks = regexp:/etc/postfix/header_checks”

Create /etc/postfix/header_checks and add the following:

/^Received:/ HOLD

Postmap it:

postmap /etc/postfix/header_checks

 

MailScanner Startup Script:

Save the following as /etc/init.d/mailscanner:

#! /bin/sh
### BEGIN INIT INFO
# Provides:          MailScanner daemon
# Required-Start:    $local_fs $remote_fs
# Required-Stop:     $local_fs $remote_fs
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Controls mailscanner instances
# Description:       MailScanner is a queue-based spam/virus filter
### END INIT INFO
# Author: Simon Walter <simon.walter@hp-factory.de>
# PATH should only include /usr/* if it runs after the mountnfs.sh script
PATH=/usr/sbin:/usr/bin:/bin:/sbin:/opt/MailScanner/bin
DESC="mail spam/virus scanner"
NAME=MailScanner
PNAME=mailscanner
DAEMON=/opt/MailScanner/bin/$NAME
STARTAS=MailScanner
SCRIPTNAME=/etc/init.d/$PNAME
CONFFILE=/opt/MailScanner/etc/MailScanner.conf
# Exit if the package is not installed
[ -x "$DAEMON" ] || exit 0
run_nice=0
stopped_lockfile=/var/lock/subsys/MailScanner.off
# Read configuration variable file if it is present
[ -r /etc/default/$PNAME ] && . /etc/default/$PNAME
# Load the VERBOSE setting and other rcS variables
. /lib/init/vars.sh
# Define LSB log_* functions.
# Depend on lsb-base (>= 3.0-6) to ensure that this file is present.
. /lib/lsb/init-functions
# sanity check for permissions
fail()
{
    echo >&2 "$0: $1"
    exit 1
}
check_dir()
{
    if [ ! -d $1 ]; then
        mkdir -p "$1" || \
            fail "directory $1: does not exist and cannot be created"
    fi
    actual="$(stat -c %U $1)"
    if [ "$actual" != "$2" ]; then
        chown -R "$2" "$1" || \
            fail "directory $1: wrong owner (expected $2 but is $actual)"
    fi
    actual="$(stat -c %G $1)"
    if [ "$actual" != "$3" ]; then
        chgrp -R "$3" "$1" || \
            fail "directory $1: wrong group (expected $3 but is $actual)"
    fi
}
user=$(echo $(awk -F= '/^Run As User/ {print $2; exit}' $CONFFILE))
group=$(echo $(awk -F= '/^Run As Group/ {print $2; exit}' $CONFFILE))
check_dir /var/spool/MailScanner       ${user:-postfix} ${group:-www-data}
check_dir /var/lib/MailScanner         ${user:-postfix} ${group:-www-data}
check_dir /var/run/MailScanner         ${user:-postfix} ${group:-www-data}
check_dir /var/lock/subsys	${user:-root}	${group:-root} #Required to Create Folder
check_dir /var/lock/subsys/MailScanner ${user:-postfix} ${group:-www-data}
#
# Function that starts the daemon/service
#
do_start()
{
        # Return
        #   0 if daemon has been started
        #   1 if daemon was already running
        #   2 if daemon could not be started
        start-stop-daemon --start --quiet --startas $STARTAS --name $NAME --test > /dev/null \
                || return 1
        start-stop-daemon --start --quiet --nicelevel $run_nice --chuid postfix:www-data --exec $DAEMON --name $NAME -- $DAEMON_ARGS \
                || return 2
        # Add code here, if necessary, that waits for the process to be ready
        # to handle requests from services started subsequently which depend
        # on this one.  As a last resort, sleep for some time.
  # Set lockfile to inform cronjobs about the running daemon
        RETVAL="$?"
        if [ $RETVAL -eq 0 ]; then
            touch /var/lock/subsys/mailscanner
            rm -f $stopped_lockfile
        fi
if [ $RETVAL -eq 0 ]; then
echo "MailScanner Started"
fi
}
#
# Function that stops the daemon/service
#
do_stop()
{
        # Return
        #   0 if daemon has been stopped
        #   1 if daemon was already stopped
        #   2 if daemon could not be stopped
        #   other if a failure occurred
        start-stop-daemon --stop --retry=TERM/30 --name $NAME
        RETVAL="$?"
        [ "$RETVAL" = 2 ] && return 2
  # Remove lockfile for cronjobs
        if [ $RETVAL -eq 0 ]; then
            rm -f /var/lock/subsys/mailscanner
            touch $stopped_lockfile
        fi
if [ $RETVAL -eq 0 ]; then
echo "MailScanner Stopped"
fi
}
#
# Function that sends a SIGHUP to the daemon/service
#
do_reload() {
        start-stop-daemon --stop --signal 1 --quiet --name $NAME
        return 0
}
case "$1" in
  start)
        [ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
        do_start
        case "$?" in
                0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
                2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
        esac
        ;;
  stop)
        [ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
        do_stop
        case "$?" in
                0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
                2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
        esac
        ;;
  restart|force-reload)
        #
        # If the "reload" option is implemented then remove the
        # 'force-reload' alias
        #
        log_daemon_msg "Restarting $DESC" "$NAME"
        do_stop
        case "$?" in
          0|1)
                do_start
                case "$?" in
                        0) log_end_msg 0 ;;
                        1) log_end_msg 1 ;; # Old process is still running
                        *) log_end_msg 1 ;; # Failed to start
                esac
                ;;
          *)
                # Failed to stop
                log_end_msg 1
                ;;
        esac
        ;;
  *)
        echo "Usage: $SCRIPTNAME {start|stop|restart|force-reload}" >&2
        exit 3
        ;;
esac
exit 0

Create Symlinks for mailscanner script to work:

chmod 755 /etc/init.d/mailscanner
ln -s ../init.d/mailscanner /etc/rc0.d/K20mailscanner
ln -s ../init.d/mailscanner /etc/rc1.d/K20mailscanner
ln -s ../init.d/mailscanner /etc/rc2.d/S20mailscanner
ln -s ../init.d/mailscanner /etc/rc3.d/S20mailscanner
ln -s ../init.d/mailscanner /etc/rc4.d/S20mailscanner
ln -s ../init.d/mailscanner /etc/rc5.d/S20mailscanner
ln -s ../init.d/mailscanner /etc/rc6.d/K20mailscanner

Start the system:

/etc/init.d/mailscanner start
/etc/init.d/postfix start

Check your logs for errors:

tail -f /var/log/mail.log

7. MailWatch Installation Instructions

This setup assumes you are using Apache v2.x and not Apache v1.x.

Before Starting, make sure that MailScanner is working before you continue with the MailWatch install!

*Notes for Ubuntu:

You must have a working MailScanner set-up and running copies of MySQL, Apache, and PHP. You must also have the Perl DBD-MySQL package installed for the Perl portions of MailScanner to utilize the MySQL database. The default php.ini set should have the following set correctly, you may want to check this:

short_open_tag = On 
safe_mode = Off 
register_globals = Off 
magic_quotes_gpc = On 
magic_quotes_runtime = Off 
session.auto_start = 0

These will be commented out you must remove the “#” to activate them:

extension=mysql.so 
extension=gd.so

 

Installation

Download the latest MailWatch release:

cd /usr/src
wget http://downloads.sourceforge.net/mailwatch/mailwatch-1.0.4.tar.gz?modtime=1178902008&big_mirror=0
tar xzvf mailwatch-1.0.4.tar.gz
cd mailwatch-1.0.4

Create the database:

mysql -p < create.sql

*Note:

You will need to modify the above as necessary for your system if you have a root password for your MySQL database (recommended!) – Debian will ask for one.

Create a MySQL user and password & set up MailScanner for SQL logging:

mysql -p
mysql> GRANT ALL ON mailscanner.* TO username@localhost IDENTIFIED BY ‘password’;

Remember the password! You need the single quotes ‘ to surround your password.

Edit MailWatch.pm and change the $db_user and $db_pass values accordingly and move MailWatch.pm:

cp MailWatch.pm /opt/MailScanner/lib/MailScanner/CustomFunctions/

Create a MailWatch Web User:

mysql mailscanner -u username -p

Enter password: password

mysql> INSERT INTO users VALUES (‘admin’,md5(‘password’),’mailscanner’,’A’,’0′,’0′,’0′,’0′,’0′);

 

Install & Configure MailWatch

From within the unpacked mailwatch directory move the directory called ‘mailscanner’ to /var/www/.

mv mailscanner/ /var/www/
cd /var/www/mailscanner

Make a temp directory:

mkdir temp
chmod g+w temp

Check the permissions of /var/www/mailscanner/images and /var/www/images/cache – they should be ug+rwx and owned by root and in the same group as the web server user.

chmod ug+rwx images
chmod ug+rwx images/cache
chown –R root:www-data *

Create conf.php by copying conf.php.example and edit the values to suit, you will need to set DB_USER and DB_PASS to the MySQL user and password that you created earlier.

define(DB_USER, 'username');
define(DB_PASS, 'password');
define(MAILWATCH_HOME, '/var/www/mailscanner/');
define(MS_CONFIG_DIR, '/opt/MailScanner/etc/');
define(MS_LIB_DIR, '/opt/MailScanner/lib/MailScanner/'); 
define(QUARANTINE_USE_FLAG, true);
define(QUARANTINE_FROM_ADDR, 'postmaster@example.com');
define(QUARANTINE_REPORT_FROM_NAME, 'SpamSnake');
define(QUARANTINE_USE_SENDMAIL, true);

 

Set-up MailScanner

Edit /opt/MailScanner/etc/MailScanner.conf and make sure the following are set:

Quarantine User = root
Quarantine Group = www-data 
Quarantine Permissions = 0660
Quarantine Whole Message = yes 
Always Looked Up Last = &MailWatchLogging

*Important: Spam Actions, High Scoring Spam Actions and No Spam Actions should also have ‘store’ as one of the keywords if you want to quarantine those items for bayes learning or viewing from within MailWatch.

 

Integrate SQL Blacklist/Whitelist (optional)

If you would like to manage the MailScanner whitelist and blacklist from within the MailWatch web interface perform the following steps.

1. Edit the MySQL connection values within the CreateList subroutine of SQLBlackWhiteList.pm to match the values you entered previous into MailWatch.pm. Both files should contain the same values. (Look for the following lines in SQLBlackWhiteList.pm and enter your own data.)

my($db_user) = 'username';
my($db_pass) = 'password';

2. Copy SQLBlackWhiteList.pm to /opt/MailScanner/lib/MailScanner/CustomFunctions/.

3. Edit MailScanner.conf and set:

Is Definitely Not Spam = &SQLWhitelist 
Is Definitely Spam = &SQLBlacklist

 

Fix to allow MailWatch to work with Postfix Inbound/Outbound Queue

Download the patch from http://www.gbnetwork.co.uk/mailscanner/postfixmail.tar.gz:

cd /usr/src
wget http://www.gbnetwork.co.uk/mailscanner/files/postfixmail.tar.gz
tar xvfz postfixmail.tar.gz
cd postfixmail
cp postfix* /var/www/mailscanner/
patch /var/www/mailscanner/functions.php functions.php.diff

 

SpamAssassin

First we need to disable the default SpamAssassin configuration file:

mv /etc/spamassassin/local.cf /etc/spamassassin/local.cf.disabled

Now let’s backup the SpamAssassin configuration file in MailScanner then edit:

cp /opt/MailScanner/etc/spam.assassin.prefs.conf /opt/MailScanner/etc/spam.assassin.prefs.conf.back

Add pyzor and razor paths to the top of /opt/MailScanner/etc/spam.assassin.prefs.conf:

pyzor_options –homedir /var/lib/MailScanner/
razor_config /var/lib/MailScanner/.razor/razor-agent.conf
dcc_path /usr/bin/dccproc

Look for these lines and change them accordingly:

bayes_ignore_header X-YOURDOMAIN-COM-MailScanner
bayes_ignore_header X-YOURDOMAIN-COM-MailScanner-SpamCheck
bayes_ignore_header X-YOURDOMAIN-COM-MailScanner-SpamScore
bayes_ignore_header X-YOURDOMAIN-COM-MailScanner-Information

“YOURDOMAIN-COM” should be replaced with whatever you used for “%org-name%” in the MailScanner.conf file. Leave the “X-” in place. This is the same orgname used in the MailScanner.conf above.

Edit the SpamAssassin /etc/spamassassin/v310.pre to enable DCC:

loadplugin Mail::SpamAssassin::Plugin::DCC

 

SpamAssassin SQL Bayes

Pre-requisities:
You’ll need the perl-DBI and perl-DBD-MySQL modules installed.

Assumptions and Variables:

SpamAssassin Bayes Database Name: sa_bayes
SpamAssassin Bayes Database UserName: sa_user
SpamAssassin Bayes Database Password: sa_password

Create the MySQL database: First of all, create a database on the server where you intend on storing the bayesian information.

mysql -u root -p
mysql> create database sa_bayes;
mysql> GRANT ALL ON sa_bayes.* TO sa_user@localhost IDENTIFIED BY ‘sa_password’;
mysql> flush privileges;

Locate the bayes_mysql.sql file:

find / -name bayes_mysql.sql
mysql -u sa_user -p sa_bayes < /path/to/bayes_mysql.sql

Make some changes to your spam.assassin.prefs.conf:

bayes_store_module Mail::SpamAssassin::BayesStore::SQL
bayes_sql_dsn DBI:mysql:sa_bayes:localhost
bayes_sql_username sa_user
bayes_sql_password sa_password
bayes_sql_override_username root

You may have to create the following to prevent an error in a lint test:

mkdir /var/www/.spamassassin

Also add this to your crontab:

30 01 * * * /usr/bin/sa-learn --force-expire --sync -p /opt/MailScanner/etc/spam.assassin.prefs.conf

Set permissions to bring it all together:

chown -R postfix:www-data /var/spool/postfix/hold
chmod -R ug+rwx /var/spool/postfix/hold

Finally make sure you restart MailScanner.

/etc/init.d/mailscanner restart

Test out the setup:

spamassassin -x -D -p /opt/MailScanner/etc/spam.assassin.prefs.conf –lint

Check for lines like:

debug: bayes: Database connection established
debug: bayes: found bayes db version 3
debug: bayes: Using userid: 2

You should see lines come up with DCC, Pyzor and Razor that say loading plugin and hopefully no errors.

Finishing up this part we need to add cron jobs that will clean/update, you probably saw the message about this after the MailScanner install script finished.

First edit conf.php and set ‘QUARANTINE_DAYS_TO_KEEP’ and change the following line in /usr/src/mailwatch-1.0.4/tools/db_clean.php and quarantine_maint.php to:

#!/usr/bin/php –q
require('/var/www/mailscanner/functions.php');

Install quarantine clean up script:

cp /usr/src/mailwatch-1.0.4/tools/quarantine_maint.php /usr/bin/quarantine_maint.php
cp /usr/src/mailwatch-1.0.4/tools/db_clean.php /usr/bin/db_clean.php
chmod +x /usr/bin/quarantine_maint.php
chmod +x /usr/bin/db_clean.php

Run

crontab -e

and add the following:

15 10 * * 2 /usr/bin/quarantine_maint.php --clean &> /dev/null
58 23 * * * /usr/bin/db_clean.php --clean &> /dev/null

Reboot the system:

reboot

Check your mail.log:

tail –f /var/log/mail.log

and you should see the following:

Jun 13 12:18:23 hoshi MailScanner[26388]: MailScanner E-Mail Virus Scanner version 4.20-3 starting…
Jun 13 12:18:24 hoshi MailScanner[26388]: Config: calling custom init function MailWatchLogging
Jun 13 12:18:24 hoshi MailScanner[26388]: Initialising database connection
Jun 13 12:18:24 hoshi MailScanner[26388]: Finished initialising database connection

Congratulations – you now have MailScanner logging to MySQL.

 

Test the MailWatch interface

Point your browser to http://<hostname>/mailscanner/ – you should be prompted for a username and password – enter the details of the MailWatch web user that you created earlier, and you should see a list of the last 50 messages processed by MailScanner.

Update the SpamAssassin Rules table

MailWatch keeps a list of all the SpamAssassin rules and descriptions which are displayed on the ‘Message Detail’ page – to show the descriptions, you need to run the updater every time you add new rules or upgrade SpamAssassin. Click on the ‘Tools/Links’ menu and select ‘Update SpamAssassin Rule Descriptions’ and click ‘Run Now’.

 

Update the GeoIP database

Change this line in /var/www/mailscanner/geoip_update.php to look like:

dbquery("LOAD DATA LOCAL INFILE

*Note: Make sure you have allow_url_fopen = On in your php.ini set.

Click on the ‘Tools/Links’ menu and select ‘Update GeoIP database’ and click ‘Run Now’.

 

Fix to allow wildcards in Whitelist/Blacklist

Add the following to the bottom of the return 1 section in your /opt/MailScanner/lib/MailScanner/CustomFunctions/SQLBlackWhiteList.pm:

return 1 if $BlackWhite->{$to}{'*@'.$fromdomain};
return 1 if $BlackWhite->{$to}{'*@*.'.$fromdomain};
return 1 if $BlackWhite->{$todomain}{'*@'.$fromdomain};
return 1 if $BlackWhite->{$todomain}{'*@*.'.$fromdomain};
return 1 if $BlackWhite->{'default'}{'*@'.$fromdomain};
return 1 if $BlackWhite->{'default'}{'*@*.'.$fromdomain};

 

Fix for Message Operations Not Finding Messages

Change the following in /var/www/mailscanner/do_message_ops.php file:

$id = $Regs[1];

to

$id = str_replace("_", ".",$Regs[1]);

 

Releasing Spam Messages

To allow MailWatch to release Spam messages without them being processed again, add 127.0.0.1 as a whitelist item in MailWatch/List interface. Make sure to restart MailScanner after configuring these options. Below is what my entry looks like.

127.0.0.1 Default Delete

 

Fix to Allow Multiple Release of Messages in Message Operations

Edit /var/www/mailscanner/do_message_ops.php and make the following changes:

case 'F':
$type='forget';
break;
case 'R':
$type='release';
break;
default:
continue;
break;

Then, find the following section and change it to look like this:

if ($type == 'release'){
if($quarantined = quarantine_list_items($id,RPC_ONLY)) {
$to = $quarantined[0]['to'];
}
echo "<tr><td><a href=\"detail.php?id=$id\">$id</a></td><td>$type</td><td>" . quarantine_release($quarantined, $itemnum, $to, RPC_ONLY) . "</td></tr>\n";
} else {
echo "<tr><td><a href=\"detail.php?id=$id\">$id</a></td><td>$type</td><td>" . quarantine_learn($items, $itemnum, $type, RPC_ONLY) . "</td></tr>\n";
}
}
echo "</TABLE>\n";
}
}
}
}
echo "  </TD>\n";

Next we edit the /var/www/mailscanner/functions.php file and change:

$fieldname[$f] = "Ops<br>S&nbsp;&nbsp;H&nbsp;&nbsp;F";

to:

$fieldname[$f] = "Ops<br>S&nbsp;&nbsp;H&nbsp;&nbsp;F&nbsp;&nbsp;R";

Next change:

array_unshift($row, "<INPUT NAME=\"OPT-REPLACEME\" TYPE=RADIO VALUE=\"S\">&nbsp;<INPUT NAME=\"OPT-REPLACEME\" TYPE=RADIO VALUE=\"H\">&nbsp;<INPUT NAME=\"OPT-REPLACEME\" TYPE=RADIO VALUE=\"F\">");

to:

array_unshift($row, "<INPUT NAME=\"OPT-REPLACEME\" TYPE=RADIO VALUE=\"S\">&nbsp;<INPUT NAME=\"OPT-REPLACEME\" TYPE=RADIO VALUE=\"H\">&nbsp;<INPUT NAME=\"OPT-REPLACEME\" TYPE=RADIO VALUE=\"F\">&nbsp;<INPUT NAME=\"OPT-REPLACEME\" TYPE=RADIO VALUE=\"R\"> ");

Next find the block with the javascript function to handle radio buttons. Add a third value like so:

echo "function SetRadios(p) {\n";
echo " var val;\n";
echo " if (p == 'S') {\n";
echo "  val = 0;\n";
echo " } else if (p == 'H') {\n";
echo "  val = 1;\n";
echo " } else if (p == 'F') {\n";
echo "  val = 2;\n";
echo " } else if (p == 'R') {\n";
echo "  val = 3;\n";
echo " } else if (p == 'C') {\n";
echo "  ClearRadios();\n";

Now, add the text for the radios:

echo "&nbsp; <a href=\"javascript:SetRadios('S')\">S</a>";
echo "&nbsp; <a href=\"javascript:SetRadios('H')\">H</a>";
echo "&nbsp; <a href=\"javascript:SetRadios('F')\">F</a>";
echo "&nbsp; <a href=\"javascript:SetRadios('R')\">R</a>";

Finally, change:

echo "<P><b>S</b> = Spam &nbsp; <b>H</b> = Ham &nbsp; <b>F</b> = Forget\n";

to:

echo "<P><b>S</b> = Spam &nbsp; <b>H</b> = Ham &nbsp; <b>F</b> = Forget &nbsp; <b>R</b> = Release\n";

 

Patch to fix autocommit error when stopping MailScanner

Edit line 80 of MailWatch.pm to look like this:

$dbh = DBI->connect("DBI:mysql:database=$db_name;host=$db_host", $db_user, $db_pass, {PrintError => 0, AutoCommit => 0});

*Note: This turns off autocommit when it connects so the commit line when it closes does not throw an error.

 

Clamd Error Fix

If you see the following error in mail.log, you have to add user clamav to the www-data group.

Dec 3 08:29:51 server1 MailScanner[3728]: Clamd::ERROR:: UNKNOWN CLAMD RETURN ./lstat() failed: Permission denied. ERROR :: /var/spool/MailScanner/incoming/3728

usermod -a -G www-data clamav

8. Install and Configure SPF

The postfix-policyd-spf-perl package depends on the Mail::SPF and the NetAddr::IP Perl modules.

We need to download postfix-policyd-spf-perl from http://www.openspf.org/Software to the /usr/src/ directory and install it to the /usr/lib/postfix/ directory like this:

cd /usr/src
wget http://www.openspf.org/blobs/postfix-policyd-spf-perl-2.007.tar.gz
tar xvfz postfix-policyd-spf-perl-2.007.tar.gz
cd postfix-policyd-spf-perl-2.007
cp postfix-policyd-spf-perl /usr/lib/postfix/policyd-spf-perl

Then we edit /etc/postfix/master.cf and add the following stanza at the end:

vi /etc/postfix/master.cf

policy unix - n n - - spawn
  user=nobody argv=/usr/bin/perl /usr/lib/postfix/policyd-spf-perl

(The leading spaces before user=nobody are important so that Postfix knows that this line belongs to the previous one!)

Then open /etc/postfix/main.cf and search for the smtpd_recipient_restrictions directive. You should have reject_unauth_destination in that directive, and right after reject_unauth_destination you add check_policy_service unix:private/policy like this:

vi /etc/postfix/main.cf

[...]
smtpd_recipient_restrictions = permit_sasl_authenticated,permit_mynetworks,reject_unauth_destination,check_policy_service unix:private/policy
[...]

or like this:

[...]
smtpd_recipient_restrictions = reject_unauth_destination,check_policy_service unix:private/policy
[...]

It is important that you specify check_policy_service AFTER reject_unauth_destination or else your system can become an open relay!

Then restart Postfix:

/etc/init.d/postfix restart

That’s it already.

 

9. Install and Configure FuzzyOcr

apt-get install netpbm gifsicle libungif-bin gocr ocrad libstring-approx-perl libmldbm-sync-perl imagemagick tesseract-ocr

Download and install the latest FuzzyOCR devel version from http://fuzzyocr.own-hero.net/wiki/Downloads:

cd /usr/src/
wget http://users.own-hero.net/~decoder/fuzzyocr/fuzzyocr-3.5.1-devel.tar.gz

Unpack FuzzyOCR and move all FuzzyOcr* files and the FuzzyOcr directory (they are all in the FuzzyOcr-3.5.1/ directory) to /etc/mail/spamassassin:

tar xvfz fuzzyocr-3.5.1-devel.tar.gz
cd FuzzyOcr-3.5.1/
mv FuzzyOcr* /etc/mail/spamassassin/
wget http://www.gbnetwork.co.uk/mailscanner/FuzzyOcr.words -O /etc/mail/spamassassin/FuzzyOcr.words

We will be storing the image hashes in a mysql database to improve on performance such that images that we have already scanned do not get scanned again as OCR is a resource intense activity.

 

Create MySQL Database

:

mysql -p < /etc/mail/spamassassin/FuzzyOcr.mysql

Setup the password:

mysql -u root -p
mysql> GRANT ALL ON FuzzyOcr.* TO fuzzyocr@localhost IDENTIFIED BY ‘password’;

 

MailWatch Fix

Do the following to prevent an error in MailWatch:

vi /etc/mail/spamassassin/FuzzyOcr.pm

Change

'use POSIX;'

to

'use POSIX qw(SIGTERM);'

 

FuzzyOcr Configuration

FuzzyOCR’s configuration file is /etc/mail/spamassassin/FuzzyOcr.cf. In that file almost everything is commented out. We open that file now and make some modifications:

vi /etc/mail/spamassassin/FuzzyOcr.cf

Put the following line into it to define the location of FuzzyOCR’s spam words file:

focr_global_wordlist /etc/mail/spamassassin/FuzzyOcr.words

/etc/mail/spamassassin/FuzzyOcr.words is a predefined word list that comes with FuzzyOCR. You can adjust it to your needs. Next change:

# Include additional scanner/preprocessor commands here:
#
focr_bin_helper pnmnorm, pnminvert, pamthreshold, ppmtopgm, pamtopnm
focr_bin_helper tesseract

to

# Include additional scanner/preprocessor commands here:
#
focr_bin_helper pnmnorm, pnminvert, convert, ppmtopgm, tesseract

Finally add/enable the following lines:

# Search path for locating helper applications
focr_path_bin /usr/local/netpbm/bin:/usr/local/bin:/usr/bin
focr_preprocessor_file /etc/mail/spamassassin/FuzzyOcr.preps
focr_scanset_file /etc/mail/spamassassin/FuzzyOcr.scansets
focr_digest_db /etc/mail/spamassassin/FuzzyOcr.hashdb
focr_db_hash /etc/mail/spamassassin/FuzzyOcr.db
focr_db_safe /etc/mail/spamassassin/FuzzyOcr.safe.db
focr_minimal_scanset 1
focr_autosort_scanset 1
focr_enable_image_hashing 3
focr_logfile /var/log/FuzzyOcr.log
#Mysql Connection#
focr_mysql_db FuzzyOcr
focr_mysql_hash Hash
focr_mysql_safe Safe
focr_mysql_user fuzzyocr
focr_mysql_pass password
focr_mysql_host localhost
focr_mysql_port 3306
focr_mysql_socket /var/run/mysqld/mysqld.sock

This is what the FuzzyOCR developers say about image hashing:

“The Image hashing database feature allows the plugin to store a vector of image features to a database, so it knows this image when it arrives a second time (and therefore does not need to scan it again). The special thing about this function is that it also recognizes the image again if it was changed slightly (which is done by spammers).”

 

Test FuzzyOCR

cd /usr/src/FuzzyOcr-3.5.1/samples
spamassassin –debug FuzzyOcr < ocr-animated.eml > /dev/null

You see the following:

[14808] info: FuzzyOcr: Found Score <9.000> for Exact Image Hash
[14808] info: FuzzyOcr: Matched [1] time(s). Prev match: 16 sec. ago
[14808] info: FuzzyOcr: Message is SPAM. Words found:
[14808] info: FuzzyOcr: “price” in 1 lines
[14808] info: FuzzyOcr: “company” in 1 lines
[14808] info: FuzzyOcr: “alert” in 1 lines
[14808] info: FuzzyOcr: “news” in 1 lines
[14808] info: FuzzyOcr: (6 word occurrences found)
[14808] dbg: FuzzyOcr: Remove DIR: /tmp/.spamassassin14808JZSvHBtmp
[14808] dbg: FuzzyOcr: Processed in 0.104555 sec.

 

10. Filtering PDF, XLS and Phishing Spam with ClamAV (Sanesecurity Signatures)

There is currently a lot of spam where the spam “information” is attached as .pdf or .xls files, sometimes also hidden inside a .zip file. While these spam mails are not easy to catch with e.g. SpamAssassin or a Bayes filter, the ClamAV virus scanner can catch them easily when it is fed with the correct signatures as ClamAV is built to scan mail attachments.

Create a folder for sanesecurity and download and give the script the proper permission.

apt-get install curl rsync
mkdir /usr/src/sanesecurity && cd /usr/src/sanesecurity
wget http://www.inetmsg.com/pub/clamav-unofficial-sigs.tar.gz
tar -zxf clamav-unofficial-sigs.tar.gz && cd clamav-unofficial-sigs-3.7
mv clamav-unofficial-sigs.sh /usr/bin
mv clamav-unofficial-sigs.conf /etc/
chmod +x /usr/bin/clamav-unofficial-sigs.sh

Edit clamav-unofficial-sigs.conf and change the following variables to match your installation:

clam_dbs="/var/lib/clamav"

The variable clamav_dbs contains the path to the directory where your ClamAV signatures are stored.

Path to clamd.pid:

clamd_pid="/var/run/clamav/clamd.pid"

Reload after update:

reload_dbs="yes"
reload_opt="kill -USR2 `cat $clamd_pid`" #Signals PID to reload dbs

Work Directory:

work_dir="/var/lib/clamav"

And once you’re done with the configuration, set the following to yes:

user_configuration_complete="yes"

Now we run the update script to check if the download works:

clamav-unofficial-sigs.sh

Now we a add the script to the root crontab to be run once a day:

crontab -e

Add the following line at the end of the root crontab:

00 04 * * * /usr/bin/clamav-unofficial-sigs.sh –c /etc/clamav-unofficial-sigs.conf &> /dev/null

 

11. Greylisting with Sqlgrey

Install sqlgrey:

apt-get install sqlgrey
touch clients_fqdn_whitelist.local && touch clients_ip_whitelist.local

Edit sqlgrey.conf to look like:

#########################
## SQLgrey config file ##
#########################
# Notes:
# - Unless specified otherwise commented settings are SQLgrey's defaults
# - SQLgrey uses a specific config file when called with -f <conf_file>
## Configuration files
conf_dir = /etc/sqlgrey
## Log level
# Uncomment to change the log level (default is normal: 2)
# nothing: O, errors only: 0, warnings: 1, normal: 2, verbose: 3, debug: 4
# loglevel = 2
## log categories can be fine-tuned,
# here are the log messages sorted by types and levels,
# (anything over the loglevel is discarded):
#
# grey     : (0) internal errors,
#            (2) initial connections, early reconnections,
#                awl matches, successful reconnections, AWL additions,
#            (3) smart decision process debug,
# whitelist: (2) whitelisted connections,
#            (3) actual whitelist hit,
#            (4) whitelists reloads,
# optin:     (3) optin/optout global result
#            (4) optin/optout SQL query results
# spam     : (2) attempts never retried,
# mail     : (1) error sending mails,
#            (4) rate-limiter debug,
# dbaccess : (0) DB errors,
#            (1) DB upgrade,
#            (2) DB upgrade details,
# martians : (2) invalid e-mail addresses,
# perf     : (2) cleanup time,
# system   : (0) error forking,
#            (3) forked children PIDs, children exits,
# conf     : (0) errors in config files, missing required file,
#            (1) warnings in config files,
#                missing optional configuration files,
#            (2) reloading configuration files,
# other    : (4) Startup cleanup
# you can set a level to O (capital o) to disable logs completely,
# but be aware that then SQLgrey can come back to haunt you...
# Provide a coma-separated "logtype:loglevel" string
# For example if you set the loglevel to 3 (verbose) but want SQLgrey to be:
# . quiet for whitelists
# . normal for greylisting
# uncomment the following line.
# log_override = whitelist:1,grey:2
# By default, log_override is empty
## Log identification
# by default this is the process name. If you define the following variable
# SQLgrey will use whatever you set it to
#log_ident = sqlgrey
## username and groupname the daemon runs as
user = sqlgrey
group = sqlgrey
## Socket
# On which socket do SQLgrey wait for queries
# use the following if you need to bind on a public IP address
# inet =
<public_ip>:2501
# default :
# inet = 2501    # bind to localhost:2501
## PID
# where to store the process PID
# pidfile = /var/run/sqlgrey.pid
## Config directory
# where to look for other configuration files (whitelists)
confdir = /etc/sqlgrey
## Greylisting delays
# If you want to be really strict (RFC-wise) use these
# This is *not* recommended, you'll have false positives
# reconnect_delay = 15    # don't allow a reconnection before 15 minutes
# max_connect_age = 2     # don't allow a reconnection after 2 hours
# default: (based on real-life experience)
# reconnect_delay = 5
# max_connect_age = 24
connect_src_throttle = 5
## Auto whitelists settings
# default is tailored for small sites
awl_age = 60
group_domain_level = 2
# For bigger sites you may want
# a smaller awl_age and a bigger group_domain_level
# awl_age = 32            # AWL must be renewed at least once a month
                          # 32 > 31 (max delay between monthly newsletters)
# group_domain_level = 10 # wait for 10 validated adresses to add a whole
                          # domain in AWL
## Database settings
# instead of Pg below use "mysql" for MySQL, "SQLite" for SQLite
# any DBD driver is allowed, but only the previous 3 have been tested
db_type = mysql
db_name = sqlgrey
# Note: the following are not used with SQLite
db_host = localhost
db_port = default
db_user = sqlgrey
db_pass = password
db_cleandelay = 1800 # in seconds, how much time between database cleanups
clean_method = sync # sync : cleanup is done in the main process,
                      #        delaying other operations
                      # async: cleanup is done in a forked process,
                      #        it won't delay mail processing
                      #        BEWARE: lockups have been reported
                      #        and are still investigated
## X-Greylist header added?
# This adds delay, whitelist and autowhitelist information in the headers
prepend = 1
## Greylisting method:
# - full   : greylist by IP address
# - classc : greylist by class C network. eg:
#            2.3.4.6 connection accepted if 2.3.4.145 did connect earlier
# - smart  : greylist by class C network unless there is no reverse lookup
#            or it looks like a home-user address
# Default is smart
# greymethod = smart
## Optin/Optout (see README.OPTINOUT for details)
# - none   : everyone is greylisted (default)
# - optin  : one must optin to have its (incoming) messages being greylisted
# - optout : one must optout to not have its messages being greylisted
optmethod = out
## SQLgrey return value.
# SQLgrey can tell Postfix to:
# - immediately reject a message with a temporary reject code
# - only do so if following rules would allow the message to pass
# The first choice will prevent Postfix from spending time evaluating
# potentially expensive rules.
# In some cases you may want following rules to be aware of the connection
# this.
#
# We can specify a different rejection strategy for the first connection
# attempt, and for early reconnections. 'immed' chooses immediate rejection
# 'delay' choose delayed rejection
#
# By default we use delay on first attempt
reject_first_attempt = immed
# Default for early reconnection is the value affected to reject_first_attempt
reject_early_reconnect = immed
## Update server
# where to get updates for whitelists
whitelists_host = sqlgrey.bouton.name
## Postmaster address
# who gets urgent notifications (DB is down for example)
# default or empty: don't send mail notifications
# admin_mail = postmaster@yourdomain.net

Create the database:

mysql -u root -p
CREATE DATABASE /*!32312 IF NOT EXISTS*/ sglgrey;
GRANT ALL ON sqlgrey.* TO sqlgrey@localhost IDENTIFIED BY ‘password’;
FLUSH PRIVILEDGES;

Edit /etc/sqlgrey/sqlgrey.conf and update the database settings sections accordingly

Add the following to smtpd_recipient_restrictions:

check_policy_service inet:127.0.0.1:2501, permit

Create

vi /usr/sbin/sqlgrey

(You can download the file from http://downloads.Kreationnext.com/spamsnake_ubuntu_9.10/sqlgrey.)

chmod +x /usr/sbin/sqlgrey

12. KAM

Create /etc/cron.daily/kam.sh with the following content:

#!/bin/bash
  
  # Original version modified by Andrew MacLachlan (andrew@gdcon.net)
  # Added additional MailScanner restarts on inital restart failure
  # Made script run silently for normal (successful) operation
  # Increased UPDATEMAXDELAY to 900 from 600
  
  # Insert a random delay up to this value, to spread virus updates round
  # the clock. 1800 seconds = 30 minutes.
  # Set this to 0 to disable it.
  UPDATEMAXDELAY=0
  if [ -f /opt/MailScanner/var/MailScanner ] ; then
  . /opt/MailScanner/var/MailScanner
  fi
  export UPDATEMAXDELAY
  
  if [ "x$UPDATEMAXDELAY" = "x0" ]; then
  :
  else
  logger -p mail.info -t KAM.cf.sh Delaying cron job up to $UPDATEMAXDELAY seconds
  perl -e "sleep int(rand($UPDATEMAXDELAY));"
  fi
  
  # JKF Fetch KAM.cf
  #echo Fetching KAM.cf...
  cd /etc/mail/spamassassin
  rm -f KAM.cf
  wget -O KAM.cf http://www.peregrinehw.com/downloads/SpamAssassin/contrib/KAM.cf > /dev/null 2>&1
  if [ "$?" = "0" ]; then
  #echo It completed and fetched something
  if ( tail -10 KAM.cf | grep -q '^#.*EOF' ); then
  # echo It succeeded so make a backup
  cp -f KAM.cf KAM.cf.backup
  else
  echo ERROR: Could not find EOF marker
  cp -f KAM.cf.backup KAM.cf
  fi
  else
  echo It failed to complete properly
  cp -f KAM.cf.backup KAM.cf
  fi
  #echo Reloading MailScanner and SpamAssassin configuration rules
  /etc/init.d/mailscanner reload > /dev/null 2>&1
  if [ $? != 0 ] ; then
  echo "MailScanner reload failed - Retrying..."
  /etc/init.d/mailscanner force-reload
  if [ $? = 0 ] ; then
  echo "MailScanner reload succeeded."
  else
  echo "Stopping MailScanner..."
  /etc/init.d/mailscanner stop
  echo "Waiting for a minute..."
  perl -e "sleep 60;"
  echo "Attemping to start MailScanner..."
  /etc/init.d/mailscanner start
  fi
  
  fi

Make it executable

chmod +x /etc/cron.daily/kam.sh

 

13. ScamNailer

Create /opt/MailScanner/bin/update_scamnailer:

(You can download the script from http://downloads.Kreationnext.com/spamsnake_ubuntu_9.10/update_scamnailer.)

Make it executable:

chmod +x /opt/MailScanner/bin/update_scamnailer

 

14. Cron jobs:

# m h  dom mon dow   command
30 0 * * * /opt/MailScanner/bin/update_phishing_sites #Update Phishing Sites
35 0 * * * /opt/MailScanner/bin/update_bad_phishing_sites #Update bad phishing sites
40 0 * * * /opt/MailScanner/bin/update_virus_scanners #Update virus scanners
@hourly /opt/MailScanner/bin/check_mailscanner #Make sure MailScanner is running
* 1 * * * /opt/MailScanner/bin/update_spamassassin #Update spamassassin
* 0 * * * /opt/MailScanner/bin/db_clean.php --clean #Clean Mailwatch DB
10 0 * * * /opt/MailScanner/bin/quarantine_maint.php --clean #Clean Quarantine
* * * * 6 /usr/sbin/update_sqlgrey_config #Update Sqlgrey Config
50 0 * * * /opt/MailScanner/bin/update_scamnailer #Update Scamnailer
* 23 * * * /usr/bin/clamav-unofficial-sigs.sh #Update Sanesecurity

 

15. Firewalling the SpamSnake with Firehol

Introduction

Firehol is a stateful iptables packet filtering firewall configurator. It is abstracted, extensible, easy and powerful. It can handle any kind of firewall, but most importantly, it gives you the means to configure it, the same way you think of it.

 

Install Firehol

apt-get install firehol

 

Firehol Settings

Edit /etc/default/firehol and change the following:

START_FIREHOL=YES

Edit /etc/firehol/firehol.conf and add the following:

version 5
# Accept all client traffic on any interface
interface any internet
protection strong
server "icmp ping ICMP ssh http https telnet webmin dns dcc echo smtp" accept
client all accept

Be sure to comment out the default configuration before applying these settings. This filters all incoming connections that are not related to the above services.
If you want to be less polite, you can drop them by adding the following after ‘protection strong’: policy drop

 

Updating RESERVED_IPS list

cd /usr/src && wget http://firehol.sf.net/firehol.tar.gz
tar xzvf firehol.tar.gz && cd firehol
mv get-iana.sh /usr/bin/get-iana.sh
chmod +x /usr/bin/get-iana.sh

Run the script to update RESERVED_IPS:

get-iana.sh

Make sure to select ‘yes’ when asked if you would like to save RESERVED_IPS to /etc/firehol/RESERVED_IPS.

Start Firehol:

/etc/init.d/firehol start

Now your server is set up to only accept connections for the services you allowed.

 

16. Apply Relay Recipients

The following directions are meant for people using Microsoft Exchange 2000 or Microsoft Exchange 2003.

This page describes how to configure your mail gateway to periodically get a list of valid recipient email addresses from your Exchange system. By doing this, you can configure your server to automatically reject any email addressed to invalid addresses. This will reduce the load on your exchange server, since it no longer has to process non-delivery reports, and it will reduce the load on your postfix server since it won’t have to perform spam and virus scanning on the message.

 

Install Dependencies

Install the perl module Net::LDAP:

perl -MCPAN -e shell
install Net::LDAP

Create the Get Email Address Script:

vi /usr/bin/getadsmtp.pl

Copy and paste the code below into this new file.

#!/usr/bin/perl -T -w
# This script will pull all users' SMTP addresses from your Active Directory
# (including primary and secondary email addresses) and list them in the
# format "user@example.com OK" which Postfix uses with relay_recipient_maps.
# Be sure to double-check the path to perl above.
# This requires Net::LDAP to be installed.  To install Net::LDAP, at a shell
# type "perl -MCPAN -e shell" and then "install Net::LDAP"
use Net::LDAP;
use Net::LDAP::Control::Paged;
use Net::LDAP::Constant ( "LDAP_CONTROL_PAGED" );
# Enter the path/file for the output
$VALID = "/etc/postfix/relay_recipients";
open VALID, ">$VALID" or die "CANNOT OPEN $VALID $!";
# Enter the FQDN of your Active Directory domain controllers below
$dc1="domaincontroller1.example.com";
$dc2="domaincontroller2.example.com";
# Enter the LDAP container for your userbase.
# The syntax is CN=Users,dc=example,dc=com
# This can be found by installing the Windows 2000 Support Tools
# then running ADSI Edit.
# In ADSI Edit, expand the "Domain NC [domaincontroller1.example.com]" &
# you will see, for example, DC=example,DC=com (this is your base).
# The Users Container will be specified in the right pane as
# CN=Users depending on your schema (this is your container).
# You can double-check this by clicking "Properties" of your user
# folder in ADSI Edit and examining the "Path" value, such as:
# LDAP://domaincontroller1.example.com/CN=Users,DC=example,DC=com
# which would be $hqbase="cn=Users,dc=example,dc=com"
# Note:  You can also use just $hqbase="dc=example,dc=com"
$hqbase="cn=Users,dc=example,dc=com";
# Enter the username & password for a valid user in your Active Directory
# with username in the form cn=username,cn=Users,dc=example,dc=com
# Make sure the user's password does not expire.  Note that this user
# does not require any special privileges.
# You can double-check this by clicking "Properties" of your user in
# ADSI Edit and examining the "Path" value, such as:
# LDAP://domaincontroller1.example.com/CN=user,CN=Users,DC=example,DC=com
# which would be $user="cn=user,cn=Users,dc=example,dc=com"
# Note: You can also use the UPN login: "user\@example.com"
$user="cn=user,cn=Users,dc=example,dc=com";
$passwd="password";
# Connecting to Active Directory domain controllers
$noldapserver=0;
$ldap = Net::LDAP->new($dc1) or
   $noldapserver=1;
if ($noldapserver == 1)  {
   $ldap = Net::LDAP->new($dc2) or
      die "Error connecting to specified domain controllers $@ \n";
}
$mesg = $ldap->bind ( dn => $user,
                      password =>$passwd);
if ( $mesg->code()) {
    die ("error:", $mesg->error_text((),"\n"));
}
# How many LDAP query results to grab for each paged round
# Set to under 1000 for Active Directory
$page = Net::LDAP::Control::Paged->new( size => 990 );
@args = ( base     => $hqbase,
# Play around with this to grab objects such as Contacts, Public Folders, etc.
# A minimal filter for just users with email would be:
# filter => "(&(sAMAccountName=*)(mail=*))"
         filter => "(& (mailnickname=*) (| (&(objectCategory=person)
                    (objectClass=user)(!(homeMDB=*))(!(msExchHomeServerName=*)))
                    (&(objectCategory=person)(objectClass=user)(|(homeMDB=*)
                    (msExchHomeServerName=*)))(&(objectCategory=person)(objectClass=contact))
                    (objectCategory=group)(objectCategory=publicFolder) ))",
          control  => [ $page ],
          attrs  => "proxyAddresses",
);
my $cookie;
while(1) {
  # Perform search
  my $mesg = $ldap->search( @args );
# Filtering results for proxyAddresses attributes
  foreach my $entry ( $mesg->entries ) {
    my $name = $entry->get_value( "cn" );
    # LDAP Attributes are multi-valued, so we have to print each one.
    foreach my $mail ( $entry->get_value( "proxyAddresses" ) ) {
     # Test if the Line starts with one of the following lines:
     # proxyAddresses: [smtp|SMTP]:
     # and also discard this starting string, so that $mail is only the
     # address without any other characters...
     if ( $mail =~ s/^(smtp|SMTP)://gs ) {
       print VALID $mail." OK\n";
     }
    }
  }
  # Only continue on LDAP_SUCCESS
  $mesg->code and last;
  # Get cookie from paged control
  my($resp)  = $mesg->control( LDAP_CONTROL_PAGED ) or last;
  $cookie    = $resp->cookie or last;
  # Set cookie in paged control
  $page->cookie($cookie);
}
if ($cookie) {
  # We had an abnormal exit, so let the server know we do not want any more
  $page->cookie($cookie);
  $page->size(0);
  $ldap->search( @args );
  # Also would be a good idea to die unhappily and inform OP at this point
     die("LDAP query unsuccessful");
}
# Add additional restrictions, users, etc. to the output file below.
#print VALID "user\@domain1.com OK\n";
#print VALID "user\@domain2.com 550 User unknown.\n";
#print VALID "domain3.com 550 User does not exist.\n";
close VALID;

Next set the permissions on the file to allow it to be executed:

chmod 500 /usr/bin/getadsmtp.pl

Edit the file to customize it for your specific domain. Since the file is read only, you will need to use :w! to save the file in vi.

1. Set $dc1 and $dc2 to the fully qualified domain names or IP addresses of 2 of your domain controllers.
2. Set $hqbase equal to the LDAP path to the container or organizational unit which holds the email accounts for which you wish to get the email addresses.
3. Set $user and $passwd to indicate which user account should be used to access this information. This account only needs to be a member of the domain, so it would be a good idea to setup an account specifically for this.

Try running the script. If it works correctly, it will create /etc/postfix/relay_recipients.

Note that if your postfix server is separated from your active directory controllers by a firewall, you will need to open TCP port 389 from the postfix server to the ADCs.

At this point, you can update your /etc/postfix/main.cf to relay_recipient_maps. You will also have to postmap the file to create the database.

getadsmtp.pl

At this point, you may want to edit /etc/postfix/relay_recipients and edit out any unwanted email addresses as this script imports everything.

 

Create the Table

postmap /etc/postfix/relay_recipients

Finally, you may want to set up a cron job to periodically update and build the /etc/postfix/relay_recipients.db file. You can set up a script called /usr/bin/update-relay-recipients.sh: (Optional)

vi /usr/bin/update-relay-recipients.sh

#!/bin/sh
/usr/bin/getadsmtp.pl
cd /etc/postfix
postmap relay_recipients

Don’t forget to make sure the following is in your /etc/postfix/main.cf file:

relay_recipient_maps = hash:/etc/postfix/relay_recipients

Make the script excutable:

chmod +x /usr/bin/update-relay-recipients.sh

Run crontab to add this script to the scheduled jobs:

crontab -e

Now add the following lines to the bottom of the file. Note that this cron job will run every day at 2:30 AM to update the database file. You may want to run yours more frequently or not depending on how often you add new email users to your system.

# syncronize relay_recipients with Active Directory addresses
30 2 * * * /usr/bin/update-relay-recipients.sh

 

17. Install Webmin (Optional):

apt-get install perl libnet-ssleay-perl libauthen-pam-perl libio-pty-perl libmd5-perl
cd /usr/src && wget http://downloads.sourceforge.net/project/webadmin/webmin/1.500/webmin_1.500_all.deb?use_mirror=iweb dpkg –install webmin_1.500_all.deb

 

18. Automatically Add A Disclaimer To Outgoing Emails With alterMIME (Optional)

This tutorial shows how to install and use alterMIME. alterMIME is a tool that can automatically add a disclaimer to emails. In this article I will explain how to install it as a Postfix filter on Ubuntu.

Installing alterMIME:

apt-get install altermime

Next we create the user filter with the home directory /var/spool/filter – alterMIME will be run as that user:

useradd -r -c “Postfix Filters” -d /var/spool/filter filter
mkdir /var/spool/filter
chown filter:filter /var/spool/filter
chmod 750 /var/spool/filter

Afterwards we create the script /etc/postfix/disclaimer which executes alterMIME. Ubuntu’s alterMIME package comes with a sample script that we can simply copy to /etc/postfix/disclaimer:

cp /usr/share/doc/altermime/examples/postfix_filter.sh /etc/postfix/disclaimer
chgrp filter /etc/postfix/disclaimer
chmod 750 /etc/postfix/disclaimer

Now the problem with this script is that it doesn’t distinguish between incoming and outgoing emails – it simply adds a disclaimer to all mails. Typically you want disclaimers only for outgoing emails, and even then not for all sender addresses. Therefore I’ve modified the /etc/postfix/disclaimer script a little bit – we’ll come to that in a minute.

Right now, we create the file /etc/postfix/disclaimer_addresses which holds all sender email addresses (one per line) for which alterMIME should add a disclaimer:

vi /etc/postfix/disclaimer_addresses

user1@example.com
user2@example.org
user3@example.net

Now we open /etc/postfix/disclaimer and modify it as follows (I have marked the parts that I’ve changed):

vi /etc/postfix/disclaimer

#!/bin/sh
# Localize these.
INSPECT_DIR=/var/spool/filter
SENDMAIL=/usr/sbin/sendmail
####### Changed From Original Script #######
DISCLAIMER_ADDRESSES=/etc/postfix/disclaimer_addresses
####### Changed From Original Script END #######
# Exit codes from <sysexits.h>
EX_TEMPFAIL=75
EX_UNAVAILABLE=69
# Clean up when done or when aborting.
trap "rm -f in.$$" 0 1 2 3 15
# Start processing.
cd $INSPECT_DIR || { echo $INSPECT_DIR does not exist; exit
$EX_TEMPFAIL; }
cat >in.$$ || { echo Cannot save mail to file; exit $EX_TEMPFAIL; }
####### Changed From Original Script #######
# obtain From address
from_address=`grep -m 1 "From:" in.$$ | cut -d "<" -f 2 | cut -d ">" -f 1`
if [ `grep -wi ^${from_address}$ ${DISCLAIMER_ADDRESSES}` ]; then
  /usr/bin/altermime --input=in.$$ \
                   --disclaimer=/etc/postfix/disclaimer.txt \
                   --disclaimer-html=/etc/postfix/disclaimer.txt \
                   --xheader="X-Copyrighted-Material: Please visit http://www.company.com/privacy.htm" || \
                    { echo Message content rejected; exit $EX_UNAVAILABLE; }
fi
####### Changed From Original Script END #######
$SENDMAIL "$@" <in.$$
exit $?

Next we need the text file /etc/postfix/disclaimer.txt which holds our disclaimer text. Ubuntu’s alterMIME package comes with a sample text that we can use for now (of course, you can modify it if you like):

cp /usr/share/doc/altermime/examples/disclaimer.txt /etc/postfix/disclaimer.txt

Finally we have to tell Postfix that it should use the /etc/postfix/disclaimer script to add disclaimers to outgoing emails. Open /etc/postfix/master.cf and add -o content_filter=dfilt: to the smtp line:

vi /etc/postfix/master.cf

#
# Postfix master process configuration file.  For details on the format
# of the file, see the master(5) manual page (command: "man 5 master").
#
# ==========================================================================
# service type  private unpriv  chroot  wakeup  maxproc command + args
#               (yes)   (yes)   (yes)   (never) (100)
# ==========================================================================
smtp      inet  n       -       -       -       -       smtpd
   -o content_filter=dfilt:
[...]

At the end of the same file, add the following two lines:

[...]
dfilt     unix    -       n       n       -       -       pipe
    flags=Rq user=filter argv=/etc/postfix/disclaimer -f ${sender} -- ${recipient}

Restart Postfix afterwards:

/etc/init.d/postfix restart

That’s it! Now a disclaimer should be added to outgoing emails sent from the addresses listed in /etc/postfix/disclaimer_addresses.

 

Congratulations

You should now have a complete working SpamSnake.

Comments

comments