Dichotomy: Python 2 and 3

The Python world suffers from the contradictions between Python 2.x and 3. The latter is considered to be a paradise-like state, in which all teething troubles and glaring inconsistencies have been fixed, and cool new developments are taking place; except, hardly anyone uses Python 3 in production environments, and Amazon only offers 2.7.

In Python 2.x, programmers battle with a ludicrous sprawl of libraries and need to decide between incompatible products like urllib and – I kid you not – urllib2 when fetching web data. If you want to run external software with 2.x, you would use check_output() from the subprocess module. In Python 3.x, however, the run() method uses different parameters and check_output() has apparently vanished.

Lambda Go

The Lambda function in Listing 2 takes the URL for the video file to be analyzed in the event parameter dictionary under the movie_url key. In a genuine production environment, no Python script can modify data in a fixed directory like data and just hope that nobody else interferes. Because Amazon Lambda functions are called in parallel, they first need to create a temporary directory for the instance using Python's tempfile module and then clean it up again when done.

To ensure that this happens if one of the functions throws an exception after a failure, the last line in a production script would be an exception handler; this was left out in the test version shown here. Line 10 of Listing 2 calls the mkdtemp() method and uses the new directory to store the data determined in intermediate steps for the next stages in the script.

Line 16 stores the video file retrieved by a web request to a URL stored in the variable movie_file; the last part of the path determines the local file name. The next step is for lines 19 to 21 to call the max-movement-lk.py script in Listing 3 (this is a Python wrapper around the C program from Listing 1) and passes in the path to the video file located in the temporary directory.

On Site

If the analysis results in a number of JPEGs of format 0001.jpg, 0002.jpg, etc., the mk-montage.py wrapper script from Listing 4 runs as of lines 24 and 25 in Listing 2. The script converts the seconds into the movie embedded in the file names to an HH::MM:SS format and feeds the old file names and the formatted labels to ImageMagick's montage tool:

montage.py -label  00:00:01 tmp/00001.jpg  [...]

The program uses this to build a contact sheet in the montage.jpg file, which ends up on Amazon's S3 storage system later, so that users can follow the link returned by the script later to download the content.

The Python script in Listing 5 acts as a wrapper around the montage binary, which is accompanied by a bunch of shared libraries in the lib directory so that the dynamically linked binary will run in the container. The LD_LIBRARY_PATH environment variable sets the search path for shared libs on the non-standard directory so that the binary actually finds them at run time.

Figure 4 shows the shared libs collected by Listing 6. Apparently, the AI program for motion analysis linked with OpenCV drags a trunk full of dependent libraries around with it. Video analysis requires sophisticated decompression technology if you want access to the raw frame data.

Figure 4: A collection of shared libs.