Cheap VPS & Xen Server

Residential Proxy Network - Hourly & Monthly Packages

Network Diagnostic Tool (NDT) On Ubuntu 7.10 Server


This guide will walk you through the setup process for implementing NDT running under Ubuntu 7.10 server. For those unfamiliar with NDT, it is a network performance testing application. NDT will measure the throughput between your server and the desktops that you run the java client from.

Some of the issues that NDT can identify on your network, include:

  • The slowest link in the end-to-end path (Dial-up modem to 10 Gbps Ethernet/OC-192)
  • The Ethernet duplex setting (full or half)
  • If congestion is limiting end-to-end throughput
  • Duplex Mismatch
  • Excessive packet loss due to faulty cables

An example of the output that you can expect is shown in the image below.

5b5635bef14b972c8b84df84a5fca3bd-3174

WARNING! Before you begin, I need to make sure that a few things are understood first. This is a very complicated process that involves recompiling and patching the kernel. As such, you should NEVER attempt this on a production server (or even an existing server, for that matter). You could very well wind up with a bricked operating system. I take absolutely no responsibility whatsoever should this go awry. That being said, I have stepped through this guide several times, since I got the initial sequence down and it has worked for me. I hope that it will do the same for you. Once again though, just to be sure, ONLY perform this on a new server that you are willing to re-install the OS on should things go bad!

With that said, let’s get started (you did read the warning, right?).

 

Install the base operating system

This guide requires starting with a fresh installation of Ubuntu 7.10 Server. I would recommend following pages one and two of the The Perfect Server – Ubuntu Gutsy Gibbon (Ubuntu 7.10) guide, as this is what I used. Be sure to install openssh-server as the guide recommends as you will be performing most of this process remotely.  My preferred SSH tool is PuTTY.

 

Post installation

Using putty, login to your newly built Ubuntu server. The first step you need to take is to edit your sources.list and comment out the cdrom entry. Run the following commands to do so:

sudo nano /etc/apt/sources.list

You will now be prompted for the password that you created during the installation. Once you have entered it, you need to find the following line and comment it out by placing a “#” in front of it.

deb cdrom:[Ubuntu-Server 7.10 _Gutsy Gibbon_ - Release i386 (20071016)]/ gutsy main restricted

When done, it should look like this:

# deb cdrom:[Ubuntu-Server 7.10 _Gutsy Gibbon_ - Release i386 (20071016)]/ gutsy main restricted

Press Ctrl-O to write out your changes and Ctrl-X to exit the nano editor.

 

Update and upgrade

Run the following two commands to make sure that your server is up to date.

sudo apt-get update

sudo apt-get upgrade

 

Prepare root access

I understand that there are some who say that root access is completely unnecessary as the sudo command should suffice. I tend to agree, but as this guide calls for patching and recompiling the kernel, I felt that it was justified.

Run the following command to set the password for root.

sudo passwd root

You will now be prompted to enter the new root password.

Once done, become root by issuing the following command.

su

 

Install pre-requisite software

You now need to install some software that is necessary for further compilations.

Run the following command to install everything that you will need.

apt-get install binutils cpp sendmail flex gcc libarchive-zip-perl libc6-dev libcompress-zlib-perl libdb4.3-dev libpcre3 libpopt-dev lynx m4 make ncftp nmap openssl perl perl-modules unzip zip zlib1g-dev autoconf automake1.9 libtool bison autotools-dev g++ build-essential libstdc++2.10-glibc2.2 rcconf libio-pty-perl libnet-ssleay-perl libauthen-pam-perl libmd5-perl gcc cpp libpcap-dev kernel-package libncurses5-dev fakeroot wget bzip2 python2.4-dev libncurses5-dev libreadline5-dev libimlib2 libimlib2-dev sun-java5-bin sun-java5-jdk sun-java5-jre csh

Download NDT, the web100 patch, userland and new kernel sources

Issue the following commands to obtain everything that you will need for a successful build.

cd /usr/src
wget http://e2epi.internet2.edu/ndt/download/ndt-3.4.4.tar.gz
wget http://www.web100.org/download/kernel/2.5.18/web100-2.5.18-200711151130.tar.gz
wget http://www.web100.org/download/userland/version1.6/web100_userland-1.6.tar.gz
wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.23.tar.bz2

Patch and compile the kernel with the web100 patch

There is an excellent guide here in regards to compiling the kernel. I have a quick walkthrough below though that should suffice.

The first step is to extract your new kernel. Issue the following command:

tar xjf linux-2.6.23.tar.bz2

Now we are going to create a symliynk for the newly extracted kernel.

ln -s linux-2.6.23 linux

Move into the kernel directory with this command.

cd /usr/src/linux

Now it is time to extract the patch. This method is probably not as clean as it could be, since you will be extracting it and re-compressing it into the BZ2 format. It was the only way I could get the patching process to run cleanly. Issue the following commands to extract and re-compress the patch file.

tar -xvzf /usr/src/web100-2.5.18-200711151130.tar.gz
bzip2 web100/web100-2.6.23-2.5.18-200711151130.patch

Before you jump right in and run a patch, it is always a good idea to perform a dry run to make sure that everything appears okay. The following command should return a successful output.

bzip2 -dc /usr/src/linux/web100/web100-2.6.23-2.5.18-200711151130.patch.bz2 | patch -p1 –dry-run

The output should be similar to what is shown below:

patching file Makefile
patching file Documentation/web100/locking.txt
patching file Documentation/web100/proc_interface.txt
patching file Documentation/web100/sysctl.txt
patching file fs/proc/Makefile
patching file fs/proc/root.c
patching file fs/proc/web100.c
patching file include/linux/netlink.h
patching file include/linux/proc_fs.h
patching file include/linux/sysctl.h
patching file include/linux/tcp.h
patching file include/net/tcp.h
patching file include/net/web100.h
patching file include/net/web100_stats.h
patching file net/ipv4/Kconfig
patching file net/ipv4/Makefile
patching file net/ipv4/sysctl_net_ipv4.c
patching file net/ipv4/tcp.c
patching file net/ipv4/tcp_cong.c
patching file net/ipv4/tcp_input.c
patching file net/ipv4/tcp_ipv4.c
patching file net/ipv4/tcp_minisocks.c
patching file net/ipv4/tcp_output.c
patching file net/ipv4/tcp_timer.c
patching file net/ipv4/web100_stats.c
patching file net/ipv6/tcp_ipv6.c

If there are no errors, then it is safe to proceed. Patch the kernel with the following command:

bzip2 -dc /usr/src/linux/web100/web100-2.6.23-2.5.18-200711151130.patch.bz2 | patch -p1

Now copy your existing configuration with this command:

cp /boot/config-`uname -r` ./.config

Now you need to activate the web100 patch before recompiling the kernel.

make menuconfig

You are looking for a section titled “IP: Web100 networking enhancements (NEW) —>” in the menu configuration. It can be found under:

networking

networking options

Follow the screen shots below if you are having problems.

acb6a2412a7f5a7ee12c79fe245d302f-3171

5929d1af85aed469d075bea91da401d4-3172

c40e61a6e603a4faf75ddf11e7c64e4b-3184

03df48cfc1a7762b443c1a5310a8d0b5-3169

Now if you chose “exit” four times, you should be prompted to save your new kernel – Do so.

d799531c53cf40dfe322fb3feb60c73c-3177

Issue the following command to not only compile your new kernel, but to also create .deb packages that you can use to install the web100 patched kernel on other Ubuntu systems (or keep as backups), so you can skip this step the next time.

make-kpkg clean
fakeroot make-kpkg –initrd –append-to-version=-web100 kernel_image kernel_headers

This process will take a long time (it could and probably will take several hours), so now is a good time to go get a cup of coffee, read a book or to walk the dog.

Okay, so the wait is over…  Let’s make sure that our files are there. Run the following two commands:

cd /usr/src
ls -l

You should see two files named:

linux-headers-2.6.23-web100-web100_2.6.23-web100-web100-10.00.Custom_i386.deb

linux-image-2.6.23-web100-web100_2.6.23-web100-web100-10.00.Custom_i386.deb

If you see them, then it is time to go ahead and install them. Issue the following commands:

dpkg -i linux-image-2.6.23-web100-web100_2.6.23-web100-web100-10.00.Custom_i386.deb
dpkg -i linux-headers-2.6.23-web100-web100_2.6.23-web100-web100-10.00.Custom_i386.deb

Once the process has completed, you need to reboot your server to make the new kernel active.

shutdown -r now

Once the system has rebooted, make sure that your new kernel is operational.

uname -r

That should return output such as this:

2.6.23-web100-web100

Install web100 userland

I am not going to get overly wordy with this as it is fairly straight forward. Simply run the following commands:

cd /usr/src/
tar -xvzf web100_userland-1.6.tar.gz
cd web100_userland-1.6
./configure –enable-python
make
make install
cd ..
rm -Rf web100_userland-1.6
rm web100_userland-1.6.tar.gz

Install NDT

It is finally time for NDT!  The true goal of this guide. Again, I will refrain from getting wordy. Just issue the following commands:

tar -xvzf ndt-3.4.4.tar.gz
cd ndt-3.4.4
./configure
make
make install

 

Edit textual preferences for your ndt page

There are some portions of the NDT web page that you will host that you might wish to change. Adding things such as your name, site name, email address, etc…

Fortunately, you no longer have to do this manually as a shell script has been included to automate the process somewhat. Issue the following commands to get started:

cd conf/
./create-html.sh

Below is a screen dump of how the process looked on my system.

Welcome to the NDT server configuration program. This program will create a custom tcpbw100.html file for your site.
Enter your site name [Internet2] :
My NDT Server  (You, of course, need to tailor this to your needs.)
Enter your site’s location [Ann Arbor – MI] : Anytown (Again, you will want to set this to your location.)
Server connection info, enter 1 for 100 Mbps, 2 for 1 Gbps [2] : 1 (I Chose “1”, but if you are on a Gigabit network, you would want to choose “2”)
Information for email trouble reporting
Enter email userid [rcarlson] :
SomeUser (Change this to your account)
Enter email domain name [internet2.edu] : localhost (Or the server that you wish to receive mail on)
Enter default subject line [Trouble report from ndt] : Enter (Enter for default unless you wish to customize it)
The base web page ‘tcpbw100.html’ has now been created. You must move this file into the ndt_DATA directory [/usr/local/ndt] created during the ‘make’ process.
Do you want to install this file now? [yes] :
Enter (This should move the file created above for you)
Enter location [/usr/local/ndt] :  Enter (Unless you have changed the default install directory)

 

Perform a little cleanup

Issue the following commands to clean up the directories and files that are no longer needed:

cd ../..
rm -Rf ndt-3.4.4
rm ndt-3.4.4.tar.gz
rm web100-2.5.18-200711151130.tar.gz

 

Start the processes

Congratulations on sticking with it! You are very nearly done. The only thing left to do is to start the fakewww and web100srv processes. Issue the following commands to do so:

/usr/local/sbin/fakewww >& /dev/null &
/usr/local/sbin/web100srv -a –ipv4 >& /dev/null &

 

Visit your web page

That’s it! You should now be ready to visit your new ndt server’s web page. Keep in mind that this is not running on Apache though, so you need to point your browser to the port that the web100srv process listens on – 7123.

An example (since I don’t know what IP address you assigned to your server), would be:

http://192.168.2.173:7123

If all has gone well, you should be presented with a page with the following applet embedded into it.

e1d4b65125fe6a81cb7c5e31a031984f-3164

Click the start button and hope for the best.  Keep in mind that this is a Java based application, so you need to have Java installed on the system that you are running the test from.

I truly hope that this works for you. Please keep in mind though that it is a fairly complicated build and was very trying to make into a guide. I almost gave up several times… Good luck.

Security Warning

Although I used the root account during this build, I do not recommend leaving the root account enabled, due to the very real security threats that it poses. To disable the root account once the build is complete and your NDT server is running, you can run the following command to disable it.

sudo passwd -l root

This will place your root account into a disabled state to prevent brute force attempts.

 

Comments

comments