Set up Amazon Web Services – Part 2
Dichotomy: Python 2 and 3
The Python world suffers from the contradictions between Python 2.x and 3. The latter is considered to be a paradise-like state, in which all teething troubles and glaring inconsistencies have been fixed, and cool new developments are taking place; except, hardly anyone uses Python 3 in production environments, and Amazon only offers 2.7.
In Python 2.x, programmers battle with a ludicrous sprawl of libraries and need to decide between incompatible products like urllib
and – I kid you not – urllib2
when fetching web data. If you want to run external software with 2.x, you would use check_output()
from the subprocess
module. In Python 3.x, however, the run()
method uses different parameters and check_output()
has apparently vanished.
Lambda Go
The Lambda function in Listing 2 takes the URL for the video file to be analyzed in the event
parameter dictionary under the movie_url
key. In a genuine production environment, no Python script can modify data in a fixed directory like data
and just hope that nobody else interferes. Because Amazon Lambda functions are called in parallel, they first need to create a temporary directory for the instance using Python's tempfile
module and then clean it up again when done.
To ensure that this happens if one of the functions throws an exception after a failure, the last line in a production script would be an exception handler; this was left out in the test version shown here. Line 10 of Listing 2 calls the mkdtemp()
method and uses the new directory to store the data determined in intermediate steps for the next stages in the script.
Line 16 stores the video file retrieved by a web request to a URL stored in the variable movie_file
; the last part of the path determines the local file name. The next step is for lines 19 to 21 to call the max-movement-lk.py
script in Listing 3 (this is a Python wrapper around the C program from Listing 1) and passes in the path to the video file located in the temporary directory.
Listing 3
max-movement-lk.py
On Site
If the analysis results in a number of JPEGs of format 0001.jpg
, 0002.jpg
, etc., the mk-montage.py
wrapper script from Listing 4 runs as of lines 24 and 25 in Listing 2. The script converts the seconds into the movie embedded in the file names to an HH::MM:SS
format and feeds the old file names and the formatted labels to ImageMagick's montage
tool:
Listing 4
mk-montage.py
montage.py -label 00:00:01 tmp/00001.jpg [...]
The program uses this to build a contact sheet in the montage.jpg
file, which ends up on Amazon's S3 storage system later, so that users can follow the link returned by the script later to download the content.
The Python script in Listing 5 acts as a wrapper around the montage
binary, which is accompanied by a bunch of shared libraries in the lib
directory so that the dynamically linked binary will run in the container. The LD_LIBRARY_PATH
environment variable sets the search path for shared libs on the non-standard directory so that the binary actually finds them at run time.
Listing 5
montage.py
Figure 4 shows the shared libs collected by Listing 6. Apparently, the AI program for motion analysis linked with OpenCV drags a trunk full of dependent libraries around with it. Video analysis requires sophisticated decompression technology if you want access to the raw frame data.
Listing 6
ldd-ls.py
« Previous 1 2 3 Next »
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Endless OS 6 has Arrived
After more than a year since the last update, the latest release of Endless OS is now available for general usage.
-
Fedora Asahi 40 Remix Available for Macs with Apple Silicon
If you've been anticipating KDE's Plasma 6 for your Apple Silicon-powered Mac, then you're in luck.
-
Red Hat Adds New Deployment Option for Enterprise Linux Platforms
Red Hat has re-imagined enterprise Linux for an AI future with Image Mode.
-
OSJH and LPI Release 2024 Open Source Pros Job Survey Results
See what open source professionals look for in a new role.
-
Proton 9.0-1 Released to Improve Gaming with Steam
The latest release of Proton 9 adds several improvements and fixes an issue that has been problematic for Linux users.
-
So Long Neofetch and Thanks for the Info
Today is a day that every Linux user who enjoys bragging about their system(s) will mourn, as Neofetch has come to an end.
-
Ubuntu 24.04 Comes with a “Flaw"
If you're thinking you might want to upgrade from your current Ubuntu release to the latest, there's something you might want to consider before doing so.
-
Canonical Releases Ubuntu 24.04
After a brief pause because of the XZ vulnerability, Ubuntu 24.04 is now available for install.
-
Linux Servers Targeted by Akira Ransomware
A group of bad actors who have already extorted $42 million have their sights set on the Linux platform.
-
TUXEDO Computers Unveils Linux Laptop Featuring AMD Ryzen CPU
This latest release is the first laptop to include the new CPU from Ryzen and Linux preinstalled.