Perl script sets up Jenkins jobs to automate builds and tests

Automatic Testing

Article from Issue 168/2014
Author(s):

Instead of configuring the Jenkins continuous integration server in the browser with mouse clicks and text input for builds, programmers can store the necessary data in the source control system and let a Perl script do the work.

Continuous integration (CI) and the associated productivity gains are now firm favorites in the development universe. Input the code into the source control system, issue a pull request, have a co-worker check it  – already the cogwheels of the CI pipeline grab the change, run it through the ever-growing test suite, and, presto, everything gets released all at once. CI servers like Jenkins [1] or TeamCity [2], which grab the latest source versions of a project and fire up a build with all tests and possible deployment steps, are enjoying growing popularity.

But spending two minutes creating a new build project in Jenkins' recalcitrant interface can wear down the enthusiasm of even the most motivated developers. As Figure 1 shows, the quest here is to find the required boxes, check them, and fill out some text fields.

Figure 1: Jenkins is currently performing a build and test of the CPAN log4perl project, which resides on GitHub.

If every Perl project requires the same command sequence to start the build process, you don't need to keep typing the same info for each new project. And, the question remains: What happens if a few dozen projects move to a new CI server, does this mean starting the insane task of typing and clicking all over again?

Minimalist Thinking

Your very own Perlmeister's vision is of a more minimalist and conventional approach, such as the one adopted by Travis-CI [3] a long time ago. In this case, a small file in the source code of the project specifies the Git repository from which Jenkins can retrieve the source code and the name under which to run the project on the Jenkins server.

The Perl script presented here then puts together the Jenkins configuration as an XML document and sends it to the server as a POST request via the Jenkins API. The result is that the project is set up without any typing.

No Docs Reading Required

How do you discover the correct format for the XML data without reading the documentation or even the Jenkins source code? Manually entering a test project in the Jenkins interface and then retrieving its XML representation reveals the secret, as shown in Figure 1. The developer here enabled Git as an option for the source code repository and then typed the URL for the GitHub project into the text box that appeared.

The project is called log4perl on the Jenkins server, and the script in Listing 2 can easily read the associated XML data via the API. The CPAN Jenkins::API module provides the project_config() method, which contacts the server on port 8080 of the localhost in the script and requests the configuration, supplying the project name.

Listing 2

jenkins-project-xml

1 #!/usr/local/bin/perl -w
2 use strict;
3 use Jenkins::API;
4
5 my $jenkins = Jenkins::API->new(
6   { base_url => 'http://localhost:8080' });
7
8 print $jenkins->project_config(
9     "log4perl" ), "\n";

First of all, however, a script like the one in Listing 1 [4] should check to see whether the Jenkins server responds to requests on the default port. If this works, Listing 2 can inquire as to what the configuration of a particular project looks like.

Listing 1

jenkins-ok

01 #!/usr/local/bin/perl -w
02 use strict;
03 use Jenkins::API;
04
05 my $jenkins = Jenkins::API->new(
06   { base_url => 'http://localhost:8080' });
07
08 if( $jenkins->check_jenkins_url() ) {
09   print "ok\n";
10 } else {
11   print "not ok\n";
12 }

Configuration Jungle

If successful, the project_config() method returns a tangled nest of XML, which – among other things – reveals the values for the URL pointing to the project's Git repository, as manually entered in the web UI (Figure 2).

Figure 2: In the XML jungle of the Jenkins configuration, you can find the URL for the project's GitHub repository, as set in the UI.

Armed with this example, a script like that shown in Listing 3 can put together a template with an XML structure and use variables (e.g., [% git_url **%]) interpreted by the CPAN Template Toolkit module to adapt them for existing projects (Figure 3).

Listing 3

jenkins-projects-sync

01 #!/usr/local/bin/perl -w
02 use strict;
03 use YAML qw( LoadFile );
04 use Jenkins::API;
05 use Template;
06 use Log::Log4perl qw(:easy);
07
08 my $projects_data_file =
09   "jenkins-projects.yml";
10 my $jenkins_url = "http://localhost:8080";
11 my $projects_tmpl_file =
12   "project-jenkins.tmpl";
13
14 Log::Log4perl->easy_init($DEBUG);
15
16 my $tt = Template->new();
17
18 my $jenkins = Jenkins::API->new(
19   { base_url => $jenkins_url });
20
21 my $data = LoadFile( $projects_data_file );
22
23 for my $project ( @$data ) {
24   my $xml = $jenkins->project_config(
25     $project->{ name } );
26
27   $tt->process( $projects_tmpl_file,
28     $project, \my $new_xml ) or
29       $tt->error;
30
31   if( $xml =~ /^<\?xml/ ) {
32     INFO "Job for $project->{ name } ",
33          "already exists.";
34
35     if( $new_xml ne $xml ) {
36       INFO
37         "Updating job $project->{ name }";
38       $jenkins->set_project_config(
39         $project->{ name },
40         $new_xml ) or die;
41     }
42     next;
43   }
44
45   INFO "Creating new job for ",
46     "$project->{ name }.";
47
48   $jenkins->create_job(
49     $project->{ name }, $new_xml ) or die;
50 }
Figure 3: In the template generated from the XML, the template processor replaces variables such as the URL to the Git repository of the project, in this case, % git_url %.

The script reads the minimum configuration of each project from the YAML data in Listing 4, consisting of the project name (e.g., log4perl) and the URL for the source code on GitHub.

Listing 4

jenkins-projects.yml

01 ---
02 -
03     name: log4perl
04     git_url: https://github.com/mschilli/log4perl.git
05 -
06     name: libwww-perl
07     git_url: https://github.com/libwww-perl/libwww-perl.git
08 -
09     name: algorithm-bucketizer
10     git_url: https://github.com/mschilli/algorithm-bucketizer-perl.git

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy Linux Magazine

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • Perl: Travis CI

    A new service on travis-ci.org picks up GitHub projects, runs new code through test suites, and notifies the owners if the build fails. Its API enables Perl scripts to gather historical build data, including who-broke-the-build tabulations.

  • This Month's News Roundup

    You can’t say we weren’t warned. We’ve known for a long time that IPv4 addresses were an endangered species.

  • Container Security

    A recent flurry of activity in the container space raises several interesting questions about security among a number of operational aspects in the enterprise environment.

  • Perl: PerlPanel

    One panel has a neat collection of applets and another has spectacular looks – but a combination of the two is rare. Now help draws nigh for the desktop: PerlPanel is extensible with do-it-yourself widgets.

  • Koha Library System

    Information technology plays a key role in modern library environments. We check out Koha, an open source integrated system that can help manage a library’s daily operations.

comments powered by Disqus

Direct Download

Read full article as PDF:

Price $2.95

News

njobs Europe
What:
Where:
Country:
Njobs Netherlands Njobs Deutschland Njobs United Kingdom Njobs Italia Njobs France Njobs Espana Njobs Poland
Njobs Austria Njobs Denmark Njobs Belgium Njobs Czech Republic Njobs Mexico Njobs India Njobs Colombia