Cheap VPS & Xen Server


Residential Proxy Network - Hourly & Monthly Packages

The Perfect Xen Setup For Debian And Ubuntu


This tutorial provides step-by-step instructions on how to install Xen (version 2; I have not tested this with version 3 yet) on a Debian Sarge (3.1) system. It should apply to Ubuntu systems with little or no modifications.

Xen lets you create guest operating systems (*nix operating systems like Linux and FreeBSD), so called “virtual machines” or domUs, under a host operating system (dom0). Using Xen you can separate your applications into different virtual machines that are totally independent from each other (e.g. a virtual machine for a mail server, a virtual machine for a high-traffic web site, another virtual machine that serves your customers’ web sites, a virtual machine for DNS, etc.), but still use the same hardware. This saves money, and what is even more important, it’s more secure. If the virtual machine of your DNS server gets hacked, it has no effect on your other virtual machines. Plus, you can move virtual machines from one Xen server to the next one.

I will use Debain Sarge for both the host OS (dom0) and the guest OS (domU). In an additional section at the end I will also show how to create a virtual local network with virtual machines, with dom0 being the router.

This howto is meant as a practical guide; it does not cover the theoretical backgrounds. They are treated in a lot of other documents in the web.

This document comes without warranty of any kind! I want to say that this is not the only way of setting up such a system. There are many ways of achieving this goal but this is the way I take. I do not issue any guarantee that this will work for you!

1 Install The Debian Host System (dom0)

You can overall follow these instructions, but with a few changes:

http://www.kreationnext.com/perfect_setup_debian_sarge
http://www.kreationnext.com/perfect_setup_debian_sarge_p2

However, it’s important that you type linux26 at the boot prompt to install a kernel 2.6 system. dom0’s FQDN in this example will be server1.example.com, so I specify server1 as Hostname and example.com as Domain name. server1.example.com’s IP address will be 192.168.0.100 in this tutorial.

When it comes to the partitioning, I select Manually edit partition table. I create the following partitions:

/boot 100 MB (Primary) (Location for the new partition: Beginning) (ext3) (Bootable flag: on <– important, otherwise your system will not boot!)
swap 1GB (Logical) (Location for the new partition: Beginning)
/ 2GB (Logical) (Location for the new partition: Beginning) (ext3)
/vserver the rest (Logical) (Location for the new partition: Beginning) (ext3)

(Side note: You can also install everything in one big partition (as described here: http://www.kreationnext.com/perfect_setup_debian_sarge), but then you have to keep in mind that the Grub stanzas I describe in this howto are slightly different. For example, when I write that I add

title Xen 2.0.6 / XenLinux 2.6.11.12-xen0
root (hd0,0)
kernel /xen.gz dom0_mem=65536
module /vmlinuz-2.6.11.12-xen0 root=/dev/hda6 ro console=tty0

to /boot/grub/menu.lst then you should probably use

title Xen 2.0.7 / XenLinux 2.6.11.12-xen0
root (hd0,0)
kernel /boot/xen.gz dom0_mem=65536
module /boot/vmlinuz-2.6.11.12-xen0 root=/dev/hda1 ro console=tty0

in that file…)

When the Debian installer prompts Choose software to install: I make no selection and go on (dom0 should run as few software as possible in order not to be vulnerable to attacks. To the outside world it will be accessible only over SSH.).

2 Configure dom0’s Network

Because the Debian Sarge installer has configured our system to get its network settings via DHCP, we have to change that now because a server should have a static IP address. Edit /etc/network/interfaces and adjust it to your needs (in this example setup I will use the IP address 192.168.0.100):

# /etc/network/interfaces — configuration file for ifup(8), ifdown(8)

# The loopback interface
auto lo
iface lo inet loopback

# The first network card – this entry was created during the Debian installation
# (network, broadcast and gateway are optional)
auto eth0
iface eth0 inet static
address 192.168.0.100
netmask 255.255.255.0
network 192.168.0.0
broadcast 192.168.0.255
gateway 192.168.0.1

Then restart your network:

/etc/init.d/networking restart

Edit /etc/resolv.conf and add some nameservers:

search server
nameserver 145.253.2.75
nameserver 193.174.32.18
nameserver 194.25.0.60

Then set dom0’s hostname:

echo server1.example.com > /etc/hostname
/bin/hostname -F /etc/hostname

3 Install Xen

There are two ways to install Xen: install the binary package from the Xen website, or compile Xen and the Xen kernels from the sources.

The first way is easier, but it has the disadvantage that the domU kernel that comes with the binary package has no support for quota and iptables, both features that I need in my virtual machines (domU). Plus, the dom0 kernel has no support for the dummy network driver, which I need at the end in the optional chapter 5 where I describe how to set up a virtual local network with virtual machines.

In chapter 3.1 I describe how to install the Xen binary package which is recommended for beginners (skip chapter 3.2 and continue with chapter 4). If you need quota and iptables in your virtual machines, then skip chapter 3.1 and continue with chapter 3.2 where I show how to install Xen from the sources.

3.1 Installing The Binary Package

Run the following commands:

apt-get remove exim4 exim4-base lpr nfs-common portmap pidentd pcmcia-cs pppoe pppoeconf ppp pppconfig
apt-get install screen ssh debootstrap python python2.3-twisted iproute bridge-utils libcurl3-dev

<– Yes
<– Yes
<– Yes

3.1.1 Install Xen

Next do this:

cd /usr/src/
wget http://www.cl.cam.ac.uk/Research/SRG/netos/xen/downloads/xen-2.0.7-install-x86_32.tgz

tar xvzf xen-2.0.7-install-x86_32.tgz
cd xen-2.0-install
./install.sh

mv /lib/tls /lib/tls.disabled

Now Xen is installed. In order to start the Xen services at boot time, do the following:

update-rc.d xend defaults 20 21
update-rc.d xendomains defaults 21 20

3.1.2 Configure The Bootloader And Reboot

Next we add the Xen kernel to Grub, our bootloader. Edit /boot/grub/menu.lst, and before the line ### BEGIN AUTOMAGIC KERNELS LIST add the following stanza:

title Xen 2.0 / XenLinux 2.6.11
kernel /xen.gz dom0_mem=64000
module /vmlinuz-2.6.11-xen0 root=/dev/hda6 ro console=tty0

Make sure that /dev/hda6 is your / partition. Keep in mind what I said about Grub and partitioning in chapter 1!

Now reboot the system:

shutdown -r now

At the boot prompt, Grub should now list Xen 2.0 / XenLinux 2.6.11 as the first kernel and boot it automatically. If your system comes up without problems, then everything is fine!

3.2 Installing From The Sources

Run the following commands:

apt-get remove exim4 exim4-base lpr nfs-common portmap pidentd pcmcia-cs pppoe pppoeconf ppp pppconfig
apt-get install iproute bridge-utils python-twisted gcc-3.3 binutils make libcurl3-dev zlib1g-dev python-dev transfig bzip2 screen ssh debootstrap libcurl3-dev libncurses5-dev
(1 line!)
cd ~
mkdir xen
cd xen
mkdir archive
cd archive

3.2.1 Install Xen

Now execute these commands:

wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.11.12.tar.bz2
wget http://www.cl.cam.ac.uk/Research/SRG/netos/xen/downloads/xen-2.0.7-src.tgz
cd ..

tar -xvzf archive/xen-2.0.7-src.tgz
tar -xvjf archive/linux-2.6.11.12.tar.bz2

cd xen-2.0
make world

make install
sh ./install.sh

Now Xen is installed. In order to start the Xen services at boot time, do the following:

update-rc.d xend defaults 20 21
update-rc.d xendomains defaults 21 20

3.2.2 Compile A New dom0 Kernel

Next we compile a new dom0 kernel with Xen-, iptables-, quota-, and dummy support. The kernel will be installed to ~/xen/install so that we can save it for other machines. From ~/xen/install we will install it to the real locations. Xen works with kernel 2.6.11, so I take the latest 2.6.11 kernel (2.6.11.12) instead of newer kernels, e.g. 2.6.14.

cd ..
mv /lib/tls /lib/tls.disabled
cp -al linux-2.6.11.12 linux-2.6.11.12-xen0
cd xen-2.0/linux-2.6.11-xen-sparse/
./mkbuildtree ../../linux-2.6.11.12-xen0/
cd ../..
echo “-xen0” > linux-2.6.11.12-xen0/localversion-xen
diff -Naur linux-2.6.11.12 linux-2.6.11.12-xen0 > archive/linux-2.6.11.12-xen0.patch

cd linux-2.6.11.12
patch -p1 < ../archive/linux-2.6.11.12-xen0.patch
cd ../
mkdir install
cd linux-2.6.11.12-xen0
cp arch/xen/configs/xen0_defconfig .config
ARCH=xen; INSTALL_PATH=../install; INSTALL_MOD_PATH=../install
export ARCH INSTALL_PATH INSTALL_MOD_PATH
make oldconfig
make menuconfig

In the kernel configuration menu that shows up we have to enable quota, iptables and the dummy network driver as modules. This is where you enable these modules:

File systems –> [*] Quota support
<M> Old quota format support
<M> Quota format v2 support

Device Drivers —> Networking support —> <M> Dummy net driver support

Device Drivers —> Networking support —> Networking options —> [*] Network packet filtering (replaces ipchains) —> IP: Netfilter Configuration —> <M> IP tables support (required for filtering/masq/NAT)

[*] means: build into the kernel statically.
<M> means: build as a kernel module.

Now we install the kernel to ~/xen/install:

make modules
make modules_install
make install

Finally, we copy the kernel to the “real” locations:

cd ../install/
cp boot/* /boot
cp -r lib/modules/2.6.11.12-xen0/ /lib/modules
cp -r usr/include/xen /usr/include

(If you are interested, this is my dom0 kernel configuration.)

3.2.3 Configure The Bootloader And Reboot

Next we add our new kernel to Grub, our bootloader. Edit /boot/grub/menu.lst, and before the line ### BEGIN AUTOMAGIC KERNELS LIST add the following stanza:

title Xen 2.0.7 / XenLinux 2.6.11.12-xen0
root (hd0,0)
kernel /xen.gz dom0_mem=65536
module /vmlinuz-2.6.11.12-xen0 root=/dev/hda6 ro console=tty0

Make sure that /dev/hda6 is your / partition. Keep in mind what I said about Grub and partitioning in chapter 1!

Now reboot the system:

shutdown -r now

At the boot prompt, Grub should now list Xen 2.0.7 / XenLinux 2.6.11.12-xen0 as the first kernel and boot it automatically. If your system comes up without problems, then everything is fine!

3.2.4 Compile A New domU Kernel

Now we compile a new domU kernel for our virtual machines. Again, we install the kernel in ~/xen/install and copy it to the real location afterwards:

cd ~/xen
cd linux-2.6.11.12
make clean
cd ../
cp -al linux-2.6.11.12 linux-2.6.11.12-xenU
cd xen-2.0/linux-2.6.11-xen-sparse/
./mkbuildtree ../../linux-2.6.11.12-xenU/
cd ../..
echo “-xenU” > linux-2.6.11.12-xenU/localversion-xen
diff -Naur linux-2.6.11.12 linux-2.6.11.12-xenU > archive/linux-2.6.11.12-xenU.patch

cd linux-2.6.11.12
patch -p1 < ../archive/linux-2.6.11.12-xenU.patch
cd ../
rm -fr install
mkdir install
cd linux-2.6.11.12-xenU
cp arch/xen/configs/xenU_defconfig .config
ARCH=xen; INSTALL_PATH=../install; INSTALL_MOD_PATH=../install
export ARCH INSTALL_PATH INSTALL_MOD_PATH
make oldconfig
make menuconfig

In the kernel comfiguration menu that shows up we have to enable quota and iptables as modules (it is important that they are modules. I could not get iptables to work in a virtual machine when I compiled it into the kernel statically!). This is where you enable these modules:

File systems –> [*] Quota support
<M> Old quota format support
<M> Quota format v2 support

Device Drivers —> Networking support —> Networking options —> [*] Network packet filtering (replaces ipchains) —> IP: Netfilter Configuration —> <M> IP tables support (required for filtering/masq/NAT)

[*] means: build into the kernel statically.
<M> means: build as a kernel module.

Now we install the kernel to ~/xen/install:

make modules
make modules_install
make install

Finally, we copy the kernel to the “real” locations:

cd ../install/
cp boot/* /boot
cp -r lib/modules/2.6.11.12-xenU/ /lib/modules
cp -r usr/include/xen /usr/include

(This is my domU kernel configuration.)

4 Create A Virtual Machine (domU)

Next we create an image of a virtual machine. It will be a basic Debian system. This image will be the template for all our virtual machines. Whenever we want to create a new virtual machine, we just copy this image, create a new Xen configuration file and boot the copy, and then we can go on and configure the copy to our needs (e.g install a mail server, web server, DNS server, etc. on it). All our images will be on the /vserver partition which should be the largest one we have.

mkdir /vserver/vm_base
mkdir /vserver/images

Now we create a 1 GB image file and a 500 MB swap image. In the end the virtual machines will have 1 GB space and 500 MB swap. These are just example values, in the real world you might want to have more space for your virtual machines (e.g. between 5 and 30 GB), so just increase the value of count to create larger images.

dd if=/dev/zero of=/vserver/images/vm_base.img bs=1024k count=1000
dd if=/dev/zero of=/vserver/images/vm_base-swap.img bs=1024k count=500

Then we format /vserver/images/vm_base.img with ext3 and vm_base-swap.img with swap:

mkfs.ext3 /vserver/images/vm_base.img

When you see the following, answer with y:

/vserver/images/mail.img is not a block special device.
Proceed anyway? (y,n) <– y

mkswap /vserver/images/vm_base-swap.img
4.1 Install A Basic Debian In The Image

In order to install a basic Debian system in our image, we mount the image, run debootstrap and a few other commands:

mount -o loop /vserver/images/vm_base.img /vserver/vm_base
debootstrap –arch i386 sarge /vserver/vm_base/ http://ftp2.de.debian.org/debian

chroot /vserver/vm_base
apt-setup

You are asked the following question:

Archive access method for apt: <– http

Then select a mirror close to you.

Afterwards, edit /etc/apt/sources.list and replace testing with stable. That’s how my /etc/apt/sources.list looks:

deb http://ftp2.de.debian.org/debian/ stable main
deb-src http://ftp2.de.debian.org/debian/ stable main

deb http://security.debian.org/ stable/updates main

Then run

apt-get update

Now we set up our locales. If we do not do this now, we will see some ugly warnings during base-config like these:

perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = “en_DE:en_US:en_GB:en”,
LC_ALL = (unset),
LANG = “en_US”
are supported and installed on your system.
perl: warning: Falling back to the standard locale (“C”).
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory

They are not serious, but ugly… So we run

apt-get install localeconf

Select locales to install (e.g. en_US ISO-8859-1) and select the standard locale (e.g. en_US).

You will be asked a few questions:

Manage locale configuration files with debconf? <– Yes
Environment settings that should override the default locale: <– do not select anything
Replace existing locale configuration files? <– Yes
Default system locale: <– e.g. en_US ISO-8859-1

Next run

base-config

You will see a menu with installation options. This is what we do:

Configure timezone
Set up users and passwords
Select and install packages (when it comes to Choose software to install:, you can choose whatever you like; I, however, choose nothing because I want to install a basic system.)
Finish configuring the base system

Don’t deal with the other menu items, you don’t need them. Then we remove nfs-common and delete /etc/hostname:

apt-get remove nfs-common
rm -f /etc/hostname

Then edit /etc/fstab. It should look like this:

/dev/hda1 / ext3 defaults 1 2
/dev/hda2 none swap sw 0 0
/dev/pts devpts gid=5,mode=620 0 0
none /dev/shm tmpfs defaults 0 0

Change /etc/network/interfaces to look like this:

auto lo
iface lo inet loopback
address 127.0.0.1
netmask 255.0.0.0

Then create /etc/hosts:

127.0.0.1 localhost.localdomain localhost

# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

Now we leave the chroot environment:

exit

Then we copy over the kernel modules to our virtual machine image and unmount the image:

cp -dpR /lib/modules/2.6.11.12-xenU /vserver/vm_base/lib/modules/
mv /vserver/vm_base/lib/tls /vserver/vm_base/lib/tls.disabled
umount /vserver/vm_base

If you get a warning like this: umount: /vserver/vm_base: device is busy don’t worry about it, it’s not important.

Now our virtual machine image template is ready!

4.2 Create And Start The First Virtual Machine

Now we create our first virtual machine, vm01, by making a copy of our template:

cp -pf /vserver/images/vm_base.img /vserver/images/vm01.img
cp -pf /vserver/images/vm_base-swap.img /vserver/images/vm01-swap.img

Then we create a Xen configuration file for vm01, /etc/xen/vm01-config.sxp:

name =”vm01″
kernel =”/boot/vmlinuz-2.6.11.12-xenU”
root =”/dev/hda1″
memory =128
disk = [‘file:/vserver/images/vm01.img,hda1,w’,’file:/vserver/images/vm01-swap.img,hda2,w’]

# network
nics=1
dhcp =”off”
ip=”192.168.0.101″
netmask=”255.255.255.0″
gateway=”192.168.0.1″
hostname=”vm01.example.com”

extra=”3″

In memory you specify the RAM you want to allocate to that virtual machine (here: 128 MB). In disk you specify which images to use and how to mount them (i.e., under which partition, e.g. hda1). This must correspond to the settings in the image’s /etc/fstab file! In the network settings we tell vm01 that its IP address is 192.168.0.101 (the main machine’s (dom0) IP address is 192.168.0.100), and what hostname it has.

If you want vm01 to start automatically at the next boot of the system, then do this:

ln -s /etc/xen/vm01-config.sxp /etc/xen/auto

Now let’s start vm01:

xm create -c /etc/xen/vm01-config.sxp

If nothing’s wrong, vm01 should come up without problems, and you should be able to login. If you installed Xen from the sources, by running iptables -L you should see that iptables is available on vm01. To leave vm01’s shell, type CTRL+] if you are at the console, or CTRL+5 if you’re using PuTTY. From the outside you should be able to connect to 192.168.0.101 via SSH.

Back on dom0’s shell, you can shutdown vm01 by running

xm shutdown vm01

Here are some other Xen commands:

xm create -c /path/to/config – Start a virtual machine.
xm shutdown <name> – Stop a virtual machine.
xm list – List all running systems.
xm console <name> – Login on a virtual machine.
xm help – List of all commands.

Now you can reboot the main system to see if vm01 comes up automatically (if you created the symlink in /etc/xen/auto):

shutdown -r now

4.3 Creating And Customizing Further Virtual Machines

You can create further virtual machines simply by copying the image template:

cp -pf /vserver/images/vm_base.img /vserver/images/vm02.img
cp -pf /vserver/images/vm_base-swap.img /vserver/images/vm02-swap.img

Then you have to create a Xen configuration file, e.g. /etc/xen/vm02-config.sxp:

name =”vm02″
kernel =”/boot/vmlinuz-2.6.11.12-xenU”
root =”/dev/hda1″
memory =64
disk = [‘file:/vserver/images/vm02.img,hda1,w’,’file:/vserver/images/vm02-swap.img,hda2,w’]

# network
nics=1
dhcp =”off”
ip=”192.168.0.102″
netmask=”255.255.255.0″
gateway=”192.168.0.1″
hostname=”vm02.example.com”

extra=”3″

Start the machine:

xm create -c /etc/xen/vm02-config.sxp

and create a symlink, if you want to start the virtual machine at boot time:

ln -s /etc/xen/vm02-config.sxp /etc/xen/auto

Now you can log into each machine, e.g. via SSH, and configure it as if it was a normal system.

You can create as many virtual machines as you like. Your hardware’s the limit!

5 Create A Virtual Local Network From The Virtual Machines (Optional)

(This chapter is optional, and what is described here can only be done if you installed Xen from the sources and compiled a dom0 kernel with iptables and the dummy network driver as modules (see chapter 3.2).)

Now let’s say you got a dedicated server in some data center that has one network card and only one IP address. Now you want to set up a web server (vm01) and a mail server (vm02) as virtual machines. If you would do it the way described above, you would need three public IP addresses (one for dom0, one for vm01, one for vm02), but you only got one. The solution is to set up a virtual local network on your server which means the dom0 has the public IP address and acts as a router (doing NAT, network address translation), and behind that router we have a local network (in this example it is the network 192.168.3.0).

This is how you do it (all these steps have to be made on dom0!):

First, we need a second network interface; it is for the local network. Since we have only one real network card (eth0) which has the public IP address, we use the dummy network driver to set up the network interface dummy0.

echo dummy >> /etc/modules

Append the following part to /etc/network/interfaces:

auto dummy0
iface dummy0 inet static
address 192.168.3.1
netmask 255.255.255.0

This will give dummy0 the IP address 192.168.3.1.

Then we have to tell Xen that it should bind the Xen bridge xen-br0 to dummy0. Therefore you have to edit /etc/xen/scripts/network. Change the line

netdev=${netdev:-eth0}

to

netdev=${netdev:-dummy0}

Of course, we have to change the network settings in /etc/xen/vm01-config.sxp and /etc/xen/vm02-config.sxp. vm01 will have the IP address 192.168.3.2, so its configuration file looks like this:

name =”vm01″
kernel =”/boot/vmlinuz-2.6.11.12-xenU”
root =”/dev/hda1″
memory =128
disk = [‘file:/vserver/images/vm01.img,hda1,w’,’file:/vserver/images/vm01-swap.img,hda2,w’]

# network
nics=1
dhcp =”off”
ip=”192.168.3.2″
netmask=”255.255.255.0″
gateway=”192.168.3.1″
hostname=”vm01.example.com”

extra=”3″

Now we have to tell dom0 that it should do NAT so that the virtual machines have internet access. We also have to tell dom0 which ports it should forward to which IP address. Therefore we create the file /etc/network/if-up.d/iptables:

#!/bin/sh

echo “1” > /proc/sys/net/ipv4/ip_forward
iptables -t nat -A POSTROUTING -s 192.168.0.0/16 -j MASQUERADE

### Port Forwarding ###
iptables -A PREROUTING -t nat -p tcp -i eth0 –dport 80 -j DNAT –to 192.168.3.2:80
iptables -A PREROUTING -t nat -p tcp -i eth0 –dport 25 -j DNAT –to 192.168.3.3:25
iptables -A PREROUTING -t nat -p tcp -i eth0 –dport 110 -j DNAT –to 192.168.3.3:110

The first two commands enable Nat’ing on dom0. In the section after ### Port Forwarding ### you put as many rules as you need. This tells dom0 to forward certain ports to certain destination ports on certain destination IP addresses. For example, the first rule tells dom0 to forward requests on port 80 (http) to port 80 on 192.168.3.2. So if you have a web server running on vm01 (192.168.3.2), then all requests on port 80 on dom0 will be forwarded to this web server. The last two rules forward ports 25 (smtp) and 110 (pop3) to our mail server vm02 (192.168.3.3).

Now we have to make that script executable:

chmod 755 /etc/network/if-up.d/iptables

Finally, we reboot the server:

shutdown -r now

After the reboot, you should have a virtual local network on your Xen system!

Whenever you need new port forwarding rules, put them at the end of /etc/network/if-up.d/iptables. And because you do not want to reboot your system whenever you need new port forwarding rules, you can run the same rule on the shell. For example, if you want to forward port 21 (ftp) to vm01, you put the rule

iptables -A PREROUTING -t nat -p tcp -i eth0 –dport 21 -j DNAT –to 192.168.3.2:21

at the end of /etc/network/if-up.d/iptables. Plus, you run this rule on the shell so that it becomes valid immediately:

iptables -A PREROUTING -t nat -p tcp -i eth0 –dport 21 -j DNAT –to 192.168.3.2:21

Comments

comments