Set up Amazon Web Services – Part 2

Dichotomy: Python 2 and 3

The Python world suffers from the contradictions between Python 2.x and 3. The latter is considered to be a paradise-like state, in which all teething troubles and glaring inconsistencies have been fixed, and cool new developments are taking place; except, hardly anyone uses Python 3 in production environments, and Amazon only offers 2.7.

In Python 2.x, programmers battle with a ludicrous sprawl of libraries and need to decide between incompatible products like urllib and – I kid you not – urllib2 when fetching web data. If you want to run external software with 2.x, you would use check_output() from the subprocess module. In Python 3.x, however, the run() method uses different parameters and check_output() has apparently vanished.

Lambda Go

The Lambda function in Listing 2 takes the URL for the video file to be analyzed in the event parameter dictionary under the movie_url key. In a genuine production environment, no Python script can modify data in a fixed directory like data and just hope that nobody else interferes. Because Amazon Lambda functions are called in parallel, they first need to create a temporary directory for the instance using Python's tempfile module and then clean it up again when done.

To ensure that this happens if one of the functions throws an exception after a failure, the last line in a production script would be an exception handler; this was left out in the test version shown here. Line 10 of Listing 2 calls the mkdtemp() method and uses the new directory to store the data determined in intermediate steps for the next stages in the script.

Line 16 stores the video file retrieved by a web request to a URL stored in the variable movie_file; the last part of the path determines the local file name. The next step is for lines 19 to 21 to call the script in Listing 3 (this is a Python wrapper around the C program from Listing 1) and passes in the path to the video file located in the temporary directory.

Listing 3


On Site

If the analysis results in a number of JPEGs of format 0001.jpg, 0002.jpg, etc., the wrapper script from Listing 4 runs as of lines 24 and 25 in Listing 2. The script converts the seconds into the movie embedded in the file names to an HH::MM:SS format and feeds the old file names and the formatted labels to ImageMagick's montage tool:

Listing 4 -label  00:00:01 tmp/00001.jpg  [...]

The program uses this to build a contact sheet in the montage.jpg file, which ends up on Amazon's S3 storage system later, so that users can follow the link returned by the script later to download the content.

The Python script in Listing 5 acts as a wrapper around the montage binary, which is accompanied by a bunch of shared libraries in the lib directory so that the dynamically linked binary will run in the container. The LD_LIBRARY_PATH environment variable sets the search path for shared libs on the non-standard directory so that the binary actually finds them at run time.

Listing 5


Figure 4 shows the shared libs collected by Listing 6. Apparently, the AI program for motion analysis linked with OpenCV drags a trunk full of dependent libraries around with it. Video analysis requires sophisticated decompression technology if you want access to the raw frame data.

Listing 6


Figure 4: A collection of shared libs.

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy Linux Magazine

Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • Perl – Video Preview

    Rather than stare at boring surveillance videos, in which nothing happens 90 percent of the time, Mike Schilli tries the OpenCV image recognition software, which automatically extracts the most exciting action sequences.

  • Python 3

    What do Python 2.x programmers need to know about Python 3?

  • Out-of-Bounds Photos

    Including out-of-bounds effects in slide shows and presentations is bound to get the undivided attention of the audience. Gimp has simple tools to create these image effects.

  • Manage Amazon S3 with s3cmd
  • PSP Movies

    You can use Linux to load movies on your a Playstation Portable (PSP) gaming console.

comments powered by Disqus

Direct Download

Read full article as PDF:

Price $2.95