Tutorials – Video Editing

A transition is a way of moving a film's narrative from one scene to the next. It differs from a cut in that a transition is gradual: The new scene wipes in from the left or fades in while the old scene fades out, etc. A cut, on the other hand, just jumps to the new scene.

Transitions in videos are like transitions in Powerpoint presentations: Unless used ironically or in Star Wars movies, they are generally considered tacky and old-fashioned. The stock transitions that come with most graphic video editors are the worst in that respect. However, custom – let's call them "vanity" – transitions can be fun, especially if they are over the top, which is what I was aiming for when I started this project.

In preparing a promotional video for an issue of Raspberry Pi Geek [1], instead of using the stock transitions, I thought it would be fun to have an animation cross the screen, uncovering the next scene in its wake. You can see the effect at the 30-second mark in Figure 1.

Figure 1: Mr. Spaceman walks across the screen, uncovering a new scene in his wake.

Vanity Transitions 101

The basic idea is you create and animate your image moving across some transparent frames. You leave the space to the left of your character transparent (each frame is a PNG image), but you fill the region to the right of the animation with a flat, primary color (usually green). Figure 2 shows what a single frame halfway through the animation would look like.

Figure 2: A frame from the animation: transparent to the left and filled with a color you will use in your chroma key effect to the right.

Then, in your video editor, you place your animation over the first scene. As the right side of your animation is transparent, the first scene will show through on the right (Figure 3). You merge them together and then use the colored area with what is called a chroma key filter to cut the green bit and make the area transparent so your second clip shows through on the right, as shown in Figure 1.

Figure 3: The first clip overlaid with the animation and its chroma key layer.

With Kdenlive [2], you cannot do this in one go: You have to create a project with the animation on top of the top video clip, render it, and then use the new rendered clip to merge it with the bottom clip using Kdenlive's chroma key filter (Figure 4).

Figure 4: In Kdenlive, you have to merge and render the animation with the top layer before you can merge it with the bottom layer.

That is because Kdenlive treats clips like a stack of cards: You cannot create a new card by fusing two cards together and then merging that with something else.

With Natron [3], things are different. Each clip is a node, and you can extract streams (shown as white arrows in Figure 5) from each clip node and merge them using different filters, which are also nodes. You can extract streams from the merged clips and merge them further. This allows you to merge the animation with the top clip (the blue node in Figure 5) and then apply a chroma key effect to the result (the green node in Figure 5), merging the composite stream with the bottom clip. The advantage is you build your effect up node by node and only have to render once to produce the final result.

Figure 5: Merging the top clip with the animation and then using a chroma key filter to merge with the bottom clip in Natron.

That got me thinking… Remember what other video-manipulation software works with streams [4]? FFmpeg [5].

FFmpeg Transitions

Suppose I want to create a vanity transition in which an animated Pacman gobbles up one scene to reveal another one below. In the spirit of this section, you can do everything using command-line tools, including the Pacman animation – if you want to try, I used ImageMagick [6] for everything, from the Pacman drawings to the generation of the 48 frames used in the transition.

For this exercise, you will need the 720p version of Sintel [7] and the 720p version of Tears of Steel [8]. Both movies were produced by the Blender Foundation and are distributed under a permissive license.

Note that the two films are not really 720p; that is, they are not 1280 pixels wide by 720 pixels high. Formatted for widescreens, they are actually 1280x534p. You can confirm this using ffprobe with the file.

The ffprobe tool gives you more useful information, like the Storage Aspect Ratio (SAR). The SAR tells you the shape of the pixels that make up each frame. A SAR of 1:1 is a perfect square. A different SAR would define an oblong shape. Before you proceed, both of your films must have the same SAR, preferably 1:1.

If necessary, you can change your movie's SAR like this:

ffmpeg -i <original_video> -filter_complex "setsar=sar=1" <corrected_video>

You may also want to cut out a one-minute clip to work with:

ffmpeg -i Sintel.2010.720p.mkv -ss 00:05:00 -t 00:01:00 Sintel_cut.mp4

Otherwise, every time you want to render something, you will have to render two complete films, and that takes a long time.

The -ss parameter tells FFmpeg the starting time from which to cut. -t tells FFmpeg the clip's length. The preceding command will give you a clip that starts at the five-minute mark in the Sintel film and that lasts for one minute.

Do the same with the Tears of Steel film. Now that you have your two clips, you can start experimenting with the chroma key filter.

First try using a static PNG image with a green section. Create a frame-sized (1280x534) image called mask.png, which is half transparent and half green. Then overlay your image on the Sintel clip like this:

ffmpeg -i Sintel_cut.mp4 -i mask.png -filter_complex "overlay" Sintel_cut_ck.mp4

The resulting Sintel_cut_ck.mp4 clip should look like Figure 6.

Figure 6: A PNG image with a green area overlaid onto the Sintel clip.

Now you can merge that clip with the Tears of Steel clip using the chroma key filter:

ffmpeg -i tears_of_steel_cut.mp4 -i Sintel_cut_ck.mp4 -filter_complex "[1:v] format=rgba, chromakey=0x00FF00:0.3 [ckout]; [0:v] [ckout] overlay; amix=inputs=2" Sintel_ToS_ck.mp4

As discussed in my previous article on FFmpeg [4], you can label input streams and then the derivative streams. This makes passing the streams from one filter to the next, and finally to the output, easier and non-ambiguous.

In this case:

• [0:v] is the first input stream (tears_of_steel_cut.mp4).
• [1:v] is the second input stream (Sintel_cut_ck.mp4). Remember that Sintel_cut_ck.mp4 is the result of overlaying the chroma key image onto the Sintel clip.
• [ckout] is the stream that is the result of applying the chromakey filter to [1:v].

In this example, after converting the second stream ([1:v]) to the RGBA color space, you pass two parameters to the chromakey filter:

1. 0x00FF00 uses the classical RRGGBB format to tell FFmpeg the color you want to use as the chroma key.

2. 0.3 is the degree of similarity with your chosen color. 0.01 will only affect pixels exactly the same color as your chosen color (so only pure green pixels), while 1.0 will affect all pixels, regardless of their color. 0.3 is just enough to avoid the "green ghosting" effect often seen around the contours of objects placed in front of a green screen.

Also notice that you use the amix filter to mix together the sound streams of both clips. If you don't use amix, the final clip will have the soundtrack from one clip or the other, but not from both.

Figure 7 shows how all this works out in the finished video.

Figure 7: Sintel and Tears of Steel merged using the chroma key filter.

That is how Kdenlive would do things. But, as mentioned above, FFmpeg is more similar to Natron. Since it can process input streams live and turn them into other streams, it can do everything in a single pass:

ffmpeg -i tears_of_steel_cut.mp4 -i Sintel_cut.mp4 -i mask.png -filter_complex "[1:v] [2:v] overlay [cmask]; [cmask] format=rgba, chromakey=0x00FF00:0.3 [ckout]; [0:v] [ckout] overlay; amix=inputs=2" Sintel_ToS_ck.mp4

In the preceding instruction, you create the [cmask] stream by overlaying stream [2:v] (mask.png) onto stream [1:v] (Sintel_cut.mp4). The chromakey is then applied to the new [cmask] stream. The resulting stream, [ckout], is overlaid onto stream [0:v] ( tears_of_steel_cut.mp4).

And now the grand finale: Using an animation's sequence of frames (as opposed to a static image) for the chroma key layer isn't substantially more complicated:

ffmpeg -i tears_of_steel_cut.mp4 -i Sintel_cut.mp4 -framerate 24 -i frames/pmframe%03d.png -filter_complex "[1:v] [2:v] overlay [cmask]; [cmask] format=rgba, chromakey=0x0000FF:0.3 [ckout]; [0:v] [ckout] overlay; amix=inputs=2" Sintel_ToS_anim.mp4

There are only two new things that are noteworthy in the preceding command:

1. The input for the third input stream does not point to an image, but to a directory called frames/ containing a sequence of images (Figure 8) with names like pmframe000.png, pmframe001.png, pmframe002.png, etc. (pmframe%03d.png means pmframe followed by three digits.)

Figure 8: A few frames from the Pacman animation used as a transition. Notice the use of blue (0x0000FF) as the chroma key color, because green clashes with some of the colors in the Sintel video layer.

2. -framerate 24 tells FFmpeg how many frames per second it must use when creating the video stream from the sequence of images.

When you play back your video, you'll see your animated Pacman crossing the frame, uncovering the next scene in your video (Figure 9).

Figure 9: Pacman chews up his scenes.

Things to Try

You can try correcting the sound fading out of the Sintel video and into the Tears of Steel video. FFmpeg has nearly as many audio filters as it does video filters, so finding one that fades from one audio stream to another shouldn't be hard. In fact, the acrossfade filter [9] looks promising.

The second thing worth tinkering with is the timing of the transition. As it stands, the transition starts right at the beginning of the clip. You could cut out the moment you want the transition to start within Sintel and Tears of Steel, create the clip with the transition, and then glue all the bits together using separate instructions. FFmpeg supplies a way to edit tasks like this on the fly. You may want to check out the trim [10] and concat [11] filters for that little bit of extra video-editing wizardry.