Creating custom transitions with FFmpeg
Tutorials – Video Editing
Most video editors supply you with a generic catalog of transitions, usually in the shape of tired wipes and fades. But what if you wanted something a little more special? FFmpeg to the rescue.
A transition is a way of moving a film's narrative from one scene to the next. It differs from a cut in that a transition is gradual: The new scene wipes in from the left or fades in while the old scene fades out, etc. A cut, on the other hand, just jumps to the new scene.
Transitions in videos are like transitions in Powerpoint presentations: Unless used ironically or in Star Wars movies, they are generally considered tacky and old-fashioned. The stock transitions that come with most graphic video editors are the worst in that respect. However, custom – let's call them "vanity" – transitions can be fun, especially if they are over the top, which is what I was aiming for when I started this project.
In preparing a promotional video for an issue of Raspberry Pi Geek [1], instead of using the stock transitions, I thought it would be fun to have an animation cross the screen, uncovering the next scene in its wake. You can see the effect at the 30-second mark in Figure 1.
Vanity Transitions 101
The basic idea is you create and animate your image moving across some transparent frames. You leave the space to the left of your character transparent (each frame is a PNG image), but you fill the region to the right of the animation with a flat, primary color (usually green). Figure 2 shows what a single frame halfway through the animation would look like.
Then, in your video editor, you place your animation over the first scene. As the right side of your animation is transparent, the first scene will show through on the right (Figure 3). You merge them together and then use the colored area with what is called a chroma key filter to cut the green bit and make the area transparent so your second clip shows through on the right, as shown in Figure 1.
With Kdenlive [2], you cannot do this in one go: You have to create a project with the animation on top of the top video clip, render it, and then use the new rendered clip to merge it with the bottom clip using Kdenlive's chroma key filter (Figure 4).
That is because Kdenlive treats clips like a stack of cards: You cannot create a new card by fusing two cards together and then merging that with something else.
With Natron [3], things are different. Each clip is a node, and you can extract streams (shown as white arrows in Figure 5) from each clip node and merge them using different filters, which are also nodes. You can extract streams from the merged clips and merge them further. This allows you to merge the animation with the top clip (the blue node in Figure 5) and then apply a chroma key effect to the result (the green node in Figure 5), merging the composite stream with the bottom clip. The advantage is you build your effect up node by node and only have to render once to produce the final result.
That got me thinking… Remember what other video-manipulation software works with streams [4]? FFmpeg [5].
FFmpeg Transitions
Suppose I want to create a vanity transition in which an animated Pacman gobbles up one scene to reveal another one below. In the spirit of this section, you can do everything using command-line tools, including the Pacman animation – if you want to try, I used ImageMagick [6] for everything, from the Pacman drawings to the generation of the 48 frames used in the transition.
For this exercise, you will need the 720p version of Sintel [7] and the 720p version of Tears of Steel [8]. Both movies were produced by the Blender Foundation and are distributed under a permissive license.
Note that the two films are not really 720p; that is, they are not 1280 pixels wide by 720 pixels high. Formatted for widescreens, they are actually 1280x534p. You can confirm this using ffprobe
with the file.
The ffprobe
tool gives you more useful information, like the Storage Aspect Ratio (SAR). The SAR tells you the shape of the pixels that make up each frame. A SAR of 1:1 is a perfect square. A different SAR would define an oblong shape. Before you proceed, both of your films must have the same SAR, preferably 1:1.
If necessary, you can change your movie's SAR like this:
ffmpeg -i <original_video> -filter_complex "setsar=sar=1" <corrected_video>
You may also want to cut out a one-minute clip to work with:
ffmpeg -i Sintel.2010.720p.mkv -ss 00:05:00 -t 00:01:00 Sintel_cut.mp4
Otherwise, every time you want to render something, you will have to render two complete films, and that takes a long time.
The -ss
parameter tells FFmpeg the starting time from which to cut. -t
tells FFmpeg the clip's length. The preceding command will give you a clip that starts at the five-minute mark in the Sintel film and that lasts for one minute.
Do the same with the Tears of Steel film. Now that you have your two clips, you can start experimenting with the chroma key filter.
First try using a static PNG image with a green section. Create a frame-sized (1280x534) image called mask.png
, which is half transparent and half green. Then overlay your image on the Sintel clip like this:
ffmpeg -i Sintel_cut.mp4 -i mask.png -filter_complex "overlay" Sintel_cut_ck.mp4
The resulting Sintel_cut_ck.mp4
clip should look like Figure 6.
Now you can merge that clip with the Tears of Steel clip using the chroma key filter:
ffmpeg -i tears_of_steel_cut.mp4 -i Sintel_cut_ck.mp4 -filter_complex "[1:v] format=rgba, chromakey=0x00FF00:0.3 [ckout]; [0:v] [ckout] overlay; amix=inputs=2" Sintel_ToS_ck.mp4
As discussed in my previous article on FFmpeg [4], you can label input streams and then the derivative streams. This makes passing the streams from one filter to the next, and finally to the output, easier and non-ambiguous.
In this case:
[0:v]
is the first input stream (tears_of_steel_cut.mp4
).[1:v]
is the second input stream (Sintel_cut_ck.mp4
). Remember thatSintel_cut_ck.mp4
is the result of overlaying the chroma key image onto the Sintel clip.[ckout]
is the stream that is the result of applying thechromakey
filter to[1:v]
.
In this example, after converting the second stream ([1:v]
) to the RGBA color space, you pass two parameters to the chromakey
filter:
1. 0x00FF00
uses the classical RRGGBB format to tell FFmpeg the color you want to use as the chroma key.
2. 0.3
is the degree of similarity with your chosen color. 0.01
will only affect pixels exactly the same color as your chosen color (so only pure green pixels), while 1.0
will affect all pixels, regardless of their color. 0.3
is just enough to avoid the "green ghosting" effect often seen around the contours of objects placed in front of a green screen.
Also notice that you use the amix
filter to mix together the sound streams of both clips. If you don't use amix
, the final clip will have the soundtrack from one clip or the other, but not from both.
Figure 7 shows how all this works out in the finished video.
That is how Kdenlive would do things. But, as mentioned above, FFmpeg is more similar to Natron. Since it can process input streams live and turn them into other streams, it can do everything in a single pass:
ffmpeg -i tears_of_steel_cut.mp4 -i Sintel_cut.mp4 -i mask.png -filter_complex "[1:v] [2:v] overlay [cmask]; [cmask] format=rgba, chromakey=0x00FF00:0.3 [ckout]; [0:v] [ckout] overlay; amix=inputs=2" Sintel_ToS_ck.mp4
In the preceding instruction, you create the [cmask]
stream by overlaying stream [2:v]
(mask.png
) onto stream [1:v]
(Sintel_cut.mp4
). The chromakey
is then applied to the new [cmask]
stream. The resulting stream, [ckout]
, is overlaid onto stream [0:v]
( tears_of_steel_cut.mp4
).
And now the grand finale: Using an animation's sequence of frames (as opposed to a static image) for the chroma key layer isn't substantially more complicated:
ffmpeg -i tears_of_steel_cut.mp4 -i Sintel_cut.mp4 -framerate 24 -i frames/pmframe%03d.png -filter_complex "[1:v] [2:v] overlay [cmask]; [cmask] format=rgba, chromakey=0x0000FF:0.3 [ckout]; [0:v] [ckout] overlay; amix=inputs=2" Sintel_ToS_anim.mp4
There are only two new things that are noteworthy in the preceding command:
1. The input for the third input stream does not point to an image, but to a directory called frames/
containing a sequence of images (Figure 8) with names like pmframe000.png
, pmframe001.png
, pmframe002.png
, etc. (pmframe%03d.png
means pmframe
followed by three digits.)
2. -framerate 24
tells FFmpeg how many frames per second it must use when creating the video stream from the sequence of images.
When you play back your video, you'll see your animated Pacman crossing the frame, uncovering the next scene in your video (Figure 9).
Things to Try
You can try correcting the sound fading out of the Sintel video and into the Tears of Steel video. FFmpeg has nearly as many audio filters as it does video filters, so finding one that fades from one audio stream to another shouldn't be hard. In fact, the acrossfade filter [9] looks promising.
The second thing worth tinkering with is the timing of the transition. As it stands, the transition starts right at the beginning of the clip. You could cut out the moment you want the transition to start within Sintel and Tears of Steel, create the clip with the transition, and then glue all the bits together using separate instructions. FFmpeg supplies a way to edit tasks like this on the fly. You may want to check out the trim [10] and concat [11] filters for that little bit of extra video-editing wizardry.
Infos
- Raspberry Pi Geek #21 promo video: https://youtu.be/nImU7NlxrMA
- Kdenlive: http://kdenlive.org/
- Natron: https://natron.fr/
- Video Wizardry by Paul Brown, Linux Magazine, issue 206, January 2018, http://www.linux-magazine.com/Issues/2018/206/Tutorials-FFmpeg
- FFmpeg: https://ffmpeg.org/
- ImageMagick: http://www.imagemagick.org/
- Sintel: https://durian.blender.org/download/
- Tears of Steel: https://mango.blender.org/download/
- acrossfade: https://ffmpeg.org/ffmpeg-filters.html#acrossfade
- trim: https://ffmpeg.org/ffmpeg-filters.html#trim
- concat: \https://ffmpeg.org/ffmpeg-filters.html#concat
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
AlmaLinux 10.0 Beta Released
The AlmaLinux OS Foundation has announced the availability of AlmaLinux 10.0 Beta ("Purple Lion") for all supported devices with significant changes.
-
Gnome 47.2 Now Available
Gnome 47.2 is now available for general use but don't expect much in the way of newness, as this is all about improvements and bug fixes.
-
Latest Cinnamon Desktop Releases with a Bold New Look
Just in time for the holidays, the developer of the Cinnamon desktop has shipped a new release to help spice up your eggnog with new features and a new look.
-
Armbian 24.11 Released with Expanded Hardware Support
If you've been waiting for Armbian to support OrangePi 5 Max and Radxa ROCK 5B+, the wait is over.
-
SUSE Renames Several Products for Better Name Recognition
SUSE has been a very powerful player in the European market, but it knows it must branch out to gain serious traction. Will a name change do the trick?
-
ESET Discovers New Linux Malware
WolfsBane is an all-in-one malware that has hit the Linux operating system and includes a dropper, a launcher, and a backdoor.
-
New Linux Kernel Patch Allows Forcing a CPU Mitigation
Even when CPU mitigations can consume precious CPU cycles, it might not be a bad idea to allow users to enable them, even if your machine isn't vulnerable.
-
Red Hat Enterprise Linux 9.5 Released
Notify your friends, loved ones, and colleagues that the latest version of RHEL is available with plenty of enhancements.
-
Linux Sees Massive Performance Increase from a Single Line of Code
With one line of code, Intel was able to increase the performance of the Linux kernel by 4,000 percent.
-
Fedora KDE Approved as an Official Spin
If you prefer the Plasma desktop environment and the Fedora distribution, you're in luck because there's now an official spin that is listed on the same level as the Fedora Workstation edition.