How Perl programmers efficiently manage asynchronous program flows

Pyramid of Doom

Article from Issue 170/2015
Author(s):

Asynchronous program flow quickly degenerates into unreadable code if it lacks an overarching concept to provide structure. Fortunately, the JavaScript community has invented some functional tricks that also help tame asynchronous Perl code.

Even experts can unintentionally build race conditions into their code. Even they can easily overlook the side effects when multithreading suddenly interrupts the flow of code. Emergency teams later have to discover the causes, which involves a huge amount of effort. After all, the error the customer reports occurs only sporadically, because it only occurs given the temporal coincidence of certain events that cannot be easily reproduced by the development department.

One Thread Is Enough

If only one thread is running – as in a JavaScript application – then there can be no such race conditions, because the CPU runs the code as written, and nothing unexpected interrupts the flow. In other words, this approach has its benefits.

For a program with only one thread to run as fast as one with multiple threads, however, it must not be allowed to sit and twiddle its thumbs while, for example, a much slower network operation is in progress. To allow this to happen, the programmer relinquishes control over selected parts of the program to an event loop, which executes the events asynchronously. Once a programmer's brain has mastered the jump to asynchronous program flows, even complicated processes are wonderfully easy to write. Before that happens, however, some hurdles must be taken.

Hipster, Help!

Some people take a long time to understand asynchronous program flow. The resurrection of JavaScript as a hipster language, especially on the server side with the fully asynchronous Node.js framework, brought to light some of the familiar and typical stumbling blocks. Some elegant solutions (e.g., PubSub, Promises, or entire frameworks like Async.js) have lately emerged to help tame the asynchronous flow of complex applications [1].

Asynchronous flow is something that newcomers also perceive as a limitation in Perl, because all of a sudden they cannot, for example, easily retrieve the content of a website with the LWP::User-Agent CPAN module and a call such as:

$ua->get("http://foo.com");

Because get() is waiting for the response from the web server, and the data is slowly trickling in, all of the application cogwheels are standing still, which is truly undesirable if you want snappy performance. Instead, everything now relies on callbacks, as in the following example with the AnyEvent framework:

http_get("http://foo.com", sub { print $_[1] } );

The program simply issues the order to get the website and returns to the main program immediately after calling the http_get() function. The event loop has accepted the job and only takes care of retrieving and collecting the data when the program again takes a break. Once everything is in place, the global event loop calls the previously attached callback with the acquired data, which it outputs using the print statement.

However, this structure really messes up the program flow of an application:

http_get($url1,
   sub {
      http_get($url2, sub {
         # ...
      });
 });

If multiple web requests occur in quick succession, whose queries involve previously retrieved results, nesting can quickly take on dimensions that are difficult to understand.

Pyramid of Doom

If not handled correctly, the callback method quickly leads to what is known as the "Pyramid of Doom." This refers to nested functions, where each callback defines another callback. At some point, even the widest screen is no longer wide enough to write out the program flow, not to mention the fact that such code is extremely difficult to read and comprehend. Who would argue with this statement given Listing 1?

Listing 1

http-get-nested

01 #!/usr/local/bin/perl -w
02 use strict;
03 use AnyEvent::HTTP;
04 use CountServer;
05
06 my $cs = CountServer->new();
07 $cs->start();
08
09 my $start_url = $cs->url() . "/test-1.txt";
10
11 http_get $start_url, sub {
12   my ($body, $hdr) = @_;
13
14   print "Got: $body\n";
15   http_get $body, sub {
16       my ($body, $hdr) = @_;
17
18       print "Got: $body\n";
19       http_get $body, sub {
20         my ($body, $hdr) = @_;
21
22         print "Got: $body\n";
23       }
24   }
25 };
26
27 my $cv = AnyEvent->condvar();
28 $cv->recv();

This simulates the problem that arises when the result of a web request flows into the next request, that is, where several web requests depend on one another. They can be processed asynchronously but need to keep to a predetermined order. To illustrate this, the script in Listing 1 [2] uses CountServer, a test server, which responds to requests on the URL http://localhost:9090 by returning another URL – in each case with a numerical path component incremented by 1:

$ curl http://localhost:9090/test-1.txt
http://localhost:9090/test-2.txt
$ curl http://localhost:9090/test-2.txt
http://localhost:9090/test-3.txt
$ curl http://localhost:9090/test-5.txt
http://localhost:9090/test-6.txt

The test client in Listing 1 starts the first asynchronous web request with the string /test-1.txt; in response, it receives the string /test-2.txt, which then sends the second request and receives test-3.txt in return. The actual output from http-get-nested in Listing 1 is:

$ ./http-get-nested
Got: http://localhost:9090/test-2.txt
Got: http://localhost:9090/test-3.txt
Got: http://localhost:9090/test-4.txt

This confirms that although the code is quite difficult to understand, it is still quite functional. The CountServer.pm code shown in Listing 2 is a very easy and simple Perl class whose start() method launches a web server from the CPAN AnyEvent::HTTPD module. The reg_cb() in line 22 lets the web server register a handler for incoming requests and thus return the path information to the requesting web client incremented by precisely 1 in each case.

Listing 2

CountServer.pm

01 use strict;
02 use URI::URL ();
03 use AnyEvent::HTTPD;
04
05 my $PORT     = 9090;
06 my $BASE_URL = "http://localhost:$PORT";
07
08 ###########################################
09 sub new {
10 ###########################################
11     bless {}, $_[0];
12 }
13
14 ###########################################
15 sub start {
16 ###########################################
17   my( $self ) = @_;
18
19   $self->{ httpd } =
20     AnyEvent::HTTPD->new( port => $PORT );
21
22   $self->{ httpd }->reg_cb(
23     "" => sub {
24       my ($hdr, $req) = @_;
25
26       my $path = $req->url()->as_string;
27       ( my $newpath = $path ) =~
28           s/(\d+)/$1 + 1/ge;
29
30       my $url = URI::URL->new( $BASE_URL );
31       $url->path( $newpath );
32
33       $req->respond ({
34         content => ['text/txt',
35         $url->as_string] } );
36       }
37   );
38 }
39
40 ###########################################
41 sub url {
42 ###########################################
43   my( $self ) = @_;
44
45   return $BASE_URL;
46 }
47
48 1;

If you're used to traditional sequenced programming and think you need to launch an external web server to test a web client, you may be scratching your head by now and suspecting some kind of sorcery. In the asynchronous world, however, it is common practice to run both server and client for testing purposes at the same time, in same program. This converts a unit test into an integration test!

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy Linux Magazine

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • WS-Addressing

    WS-Addressing is a standard that enables flexible communication between web services.

  • AJAX

    AJAX offers a fast and efficient approach for building interactive websites. We’ll show you how to call upon the powers of AJAX for your own web creations.

  • WebRTC Protocol

    The WebRTC protocol converts your web browser into a communications center, supporting video chat over a peer-to-peer connection without the need for helper apps or browser plugins.

  • Perl: Network Monitoring

    To discover possibly undesirable arrivals and departures on their networks, a Perl daemon periodically stores the data from Nmap scans and passes them on to Nagios via a built-in web interface.

  • HTML5 Offline

    An offline cache in your browser and a bit of HTML5 acrobatics combine for interactive web applications that keep working even when the Internet connection breaks down.

comments powered by Disqus

Direct Download

Read full article as PDF:

Price $2.95

News

njobs Europe
What:
Where:
Country:
Njobs Netherlands Njobs Deutschland Njobs United Kingdom Njobs Italia Njobs France Njobs Espana Njobs Poland
Njobs Austria Njobs Denmark Njobs Belgium Njobs Czech Republic Njobs Mexico Njobs India Njobs Colombia