Cheap VPS & Xen Server

Residential Proxy Network - Hourly & Monthly Packages

Continuous Deployment With Jenkins And Rex


In this tutorial I will show you how to do continuous deployment with jenkins as a ci tool and Rex as a deployment tool. Together, we will set up Jenkins, the build and test process, a Subversion repository and our example Perl App.

This tutorial can be used with Java-, PHP-, Rails-, … projects as well. But for simplicity I’ve chosen Perl.

This howto focuses on Ubuntu LTS 11.04 but it can also be done with other Linux Distributions. For Jenkins you just need a Java Environment and for Rex you just need Perl 5.8 (i recommend >=5.10) and some Perl Modules (see http://rexify.org/).

Read http://rexify.org/howtos/start.html to get a short overview of what Rex is and how it works.

 

Preparation

You need the following systems. It is, for sure, possible to build everything on one system, but in this tutorial I will use for every service a seperate system.

  • Continuous Integration Server (short: CI Server). This is the System where Jenkins and the Tests will run on. (Ubuntu 11.04 LTS)
  • Subversion Server (Ubuntu 11.04 LTS)
  • 2 Test Webservers. This will be the systems where we will deploy our test project to. (Ubuntu 11.04 LTS)
  • Your Workstation (doesn’t matter what Operating System. I prefer Ubuntu, Debian or Mac OS X)

 

Installing Rex

Install it on: Your Workstation, CI Server

There are multiple prebuilt packages available for Rex. Just have a look at http://rexify.org/get/.

You can install Rex via CPAN as well:

sudo sh -c “curl -L cpanmin.us | perl – Rex”
sudo sh -c “cpanp -i –skiptest Rex::Apache::Deploy”

To build Rex you need the following dependencies:

  • build-essential
  • libexpat1-dev
  • libssh2-1-dev
  • zlib1g-dev
  • libssl-dev

Don’t forget to install Rex and Rex::Apache::Deploy on your Jenkins server as well. You need at least Rex 0.19.0 and Rex::Apache::Deploy 0.7.0.

 

Installing Tomcat

Install it on: CI Server

At first we will install Jenkins. Fire up a shell and use apt-get to install tomcat6 and tomcat6-admin. If you want, you can use this howto that shows you how to install Tomcat with Rex and jump to Step 2 “Deploying Jenkins”.

sudo apt-get install tomcat6 tomcat6-admin

Now open the file /etc/tomcat6/tomcat-users.xml and replace the content with the following.

 <?xml version='1.0' encoding='utf-8'?>
 <tomcat-users>
    <role rolename="manager"/>
    <user username="manager" password="passw0rd" roles="manager"/>
 </tomcat-users>

Now restart Tomcat.

sudo /etc/init.d/tomcat6 restart

 

Deploying Jenkins

Install it on: CI Server

Download Jenkins from http://mirrors.jenkins-ci.org/war/latest/jenkins.war.

wget http://mirrors.jenkins-ci.org/war/latest/jenkins.war

First your need to create the directory /usr/share/tomcat6/.jenkins and change its owner to tomcat6.

sudo mkdir /usr/share/tomcat6/.jenkins
sudo chown tomcat6: /usr/share/tomcat6/.jenkins

Now open up a webbrowser and go to http://your-ci-server:8080/manager/html and select jenkins.war to deploy.

deploy_war

After deploying Jenkins go to http://ci-server/jenkins/.

Okay, if you see the following screenshot everything is working.

welcome_to_jenkins

Next you will create a Subversion repository where the example project gets managed.

Example Repository

Install it on: Subversion Server

sudo apt-get install subversion libapache2-svn apache2

After you’ve installed the packages edit the file /etc/apache2/mods-enabled/dav_svn.conf

 <Location /svn>
    # enable svn
    DAV svn
    # set parent path to allow multiple repositories
    SVNParentPath /var/lib/svn
 </Location>

Save the file and create the repository.

sudo mkdir /var/lib/svn
sudo cd /var/lib/svn
sudo svnadmin create webapp
sudo chown -R www-data: .
sudo /etc/init.d/apache2 restart

 

Example Project

Okay, after you’ve created your repository, switch back to your workstation, check it out and create a small Project. I’ve chosen a Mojolicious Project (http://mojolicio.us), because it is easy to setup and to handle.

If you know how to set up Jenkins to work with other projects, for example PHP, RubyOnRails, Java, … feel free to use a own project.

First install Mojolicious:

sudo sh -c “curl -L cpanmin.us | perl – Mojolicious”

Install Mojolicious on your CI Server and Webserver, too.

After installing create a mojo app.

svn co http://your-svn-server/svn/webapp
cd webapp
mojo generate app MySite
cd my_site
mojo generate makefile
sed -ie ‘s/MyApp/MySite/g’ Makefile.PL
sed -ie ‘s/my_app/my_site/g’ Makefile.PL
rm Makefile.PLe

Now create a configuration file for the build-in webserver Hypnotoad. Create a new file hypnotoad.conf and put the following lines in it.

 {
    listen   => ['http://*:80'],
    workers  => 5,
    pid_file => '/var/run/hypnotoad.pid',
    user     => 'www-data'
 };

Okay that is enough for the first try. You can view your application by starting the development server.

./script/my_site daemon

This will start a development server on port 3000 on your workstation.

Add everything to subversion and commit it.

cd ../..
svn add my_site
svn ci -m “inital commit”

 

Installing TAP::Harness::JUnit

Install it on: CI Server

To get Jenkins compatible test outputs we need to install another perl module. This module is called TAP::Harness::JUnit.

Don’t forget the dependencies

sudo apt-get install curl build-essential
sudo sh -c “curl -L cpanmin.us | perl – TAP::Harness::JUnit”

Configuring Jenkins

A word at the start: Be carefull, don’t use spaces in the project name!

Start your Browser and point it to your Jenkins installation. Create a new “Free Style” Project.

create_freestyle_mysite

Check “Subversion” at the Source-Code-Management category. Add the Repository URL. http://your-subversion-server/svn/webapp

subversion

Now add a new build step and select “Execute Shell”. Paste the following command to the text field.

shell_run_test_mysite

cd my_site; prove -I ./lib -v –harness=TAP::Harness::JUnit

And advise Jenkins to publish the JUnit test prove created. Check “Publish JUnit test result report” in the Post-build Actions section. And check “Retain long standard output/error”, too. Put the following line into the textfield.

publish_junit_mysite

my_site/junit_output.xml

Save the project.

Now you can schedule a test build by clicking on the “Schedule a build” button.

run_job

If you refresh the Jenkins page after a few seconds, you will have a successfull testbuild.

Okay, now we need to create a Job to deploy our application on our Webservers.

Go to the Jenkins start page and select “new Job”. And, as before, choose the “Free Style” project.

create_freestyle_rex

Use the same options as before (but use an other Job name). Select “Execute shell” in the “Build” Section and use the following command.

shell_run_rex

/usr/local/bin/rex -o JUnit jenkins

If you’ve used the packages installing rex, you have to use the following command.

/usr/bin/rex -o JUnit jenkins

You alse need to check “Build after other projects are built” in “Build Triggers” section.

build_triggers_rex

This will guarantee that the Deploy Job will be started on a successfull build.

Check “Publish JUnit test result report” and this time use the filename “junit_output.xml”.

publish_junit_rex

Save the job.

 

Create a Rexfile

Now go back to your project and create a Rexfile in the top of it.

 use strict;
 use warnings;
 
 # we want to use transactions to rollback on deploy failures
 use Rex::Transaction;
 
 use Rex::Apache::Build;
 use Rex::Apache::Deploy "Symlink";
 
 # we login as root with password test
 user "root";
 password "test";
 pass_auth;
  
 # configure 2 webservers. You just want to use one here.
 group frontend => "www01", "www02";
  
 # we need to get the current revision, so the build task
 # can create the right deploy package.
 get_version_from "svn-info.tmp", qr{Revision: (\d+)};
 
 task "build", sub {
 
    # first get the current svn revision
    run "svn info >svn-info.tmp";
 
    # build a package
    build "mysite",
       path => "my_site",
       exclude => [".svn", "log"];
 
 };
 
 # this is the directory where rex will upload the deployment package
 # and extract it
 deploy_to "/var/deploy";
 document_root "/var/www/html";
   
 # generate the right directory to extract the deploy package.
 generate_deploy_directory sub {
    my ($file) = @_;
    $file =~ m/-(\d+)\.tar\.gz$/;
    return $1;
 };
 
 task "deploy", group => "frontend", sub {
 
    # on failure, roll back to the old version
    my $old_live = get_live_version;
    on_rollback {
       say "Rolling back to $old_live\n";
       switch_to_version($old_live);
       reload();
    };
 
    deploy "mysite";
 
    # create dependencies on the tasks "test" and "reload"
    needs "test";
    needs "reload";
 
 };
 
 task "test", group => "frontend", sub {
    run "cd /var/www/html; perl Makefile.PL; make; make test;";
    if($? != 0) {
       die("Error testing deployment.");
    }

    # Add more tests, for example some Selenium tests
 };
 
 task "reload", group => "frontend", sub {
    # quit hynotoad
    eval { kill cat "/var/run/hypnotoad.pid"; };
 
    # and restart it
    run "cd /var/www/html; hypnotoad script/my_site";
 };
 
 task "jenkins", sub {
    transaction {
       do_task [ qw/build deploy/ ];
    };
 };

Commit the Rexfile to the repository.

svn add Rexfile
svn ci -m “Added Rexfile” Rexfile

After you created your Rexfile, login to your CI server and install subversion.

apt-get install subversion

 

Preparing the Webserver

On your Webserver, we will use the build-in webserver of Mojolicious. So install Mojolicious on the Webserver.

sudo apt-get install curl build-essential
sudo sh -c “curl -L cpanmin.us | perl – Mojolicious”

Create the directories /var/deploy and /var/www. The first is the directory where all the deployments will be placed, the latter is the directory where your Webapp will run. Rex will create symlinks from your DocumentRoot pointing to the version currently live. So it is easy to rollback to older versions if needed.

mkdir /var/deploy
mkdir /var/www

Now your’re ready to start the deploy job in jenkins.

After running the job (without failures), you can point your browser to http://your-web-server/ and the Mojolicious default 404 site should pop up.

 

Playing with Jenkins

Well, after we have set up everything, lets just add an index page to our Mojolicious Application.

To do so, open the file lib/MySite.pm and add a new route.

$r->route('/')->to('root#index');

Save the file and create a new file lib/MySite/Root.pm and paste the following content into it.

 package MySite::Root;
 use Mojo::Base 'Mojolicious::Controller';
 
 # This action will render a template
 sub index {
    my $self = shift;
  
    # Render template "root/index.html.ep" with message
    $self->render(msg => "Welcome here.");
 }
                                                                                                             
 1;

And create the file templates/root/index.html.ep.

 <html>
   <head>
      <meta http-equiv="content-type" content="text/html; charset=utf-8">
   
      <title>Testpage</title>
      
   </head>
   <body>
      <h1>Welcome Mojo</h1>
      <p><%= $msg%>
   </body>
 </html>

Save the file and add it to version control.

svn add lib/MySite/Root.pm templates/root
svn ci -m “added root page” templates/root lib/MySite/Root.pm lib/MySite.pm

And now, start the MySite Jenkins job.

As you see, if you reload your browser after the jobs ran, the new page is deployed.

 

Now it’s up to you

Now it’s up to you to improve these basic steps. For example add some commit- check scripts to validate coding standards before the commit is accepted by subversion. Or, to prohibit new code from being checked in if a build fails. The last one is, imho, a very usefull and important thing to get a first feedback loop and to fix builds fast.

Before you try to deploy to your livesystems, build some testsytems and add another Job to Jenkins to deploy to the testsystem first. And on success to the live systems. You can use Rex Environments to archive this easily.

Comments

comments