Tutorials – Natron

In last month's issue [1], we had a look at Natron [2] and saw some basic things we could do with it. But Natron does much more than basic. In fact, doing complex effects and composites involving several tracks and techniques is what Natron does best.

To illustrate what you can achieve with Natron, read on for a couple of practical examples.

Transition

This effect lets you use an animation to transition from one clip to another. The animation in this case is a running person on a green background. The effect you want to achieve is having the character run from right to left, dragging the second clip in over the first one [3].

If you are a regular reader of Linux Magazine, this should be familiar to you. We did this effect using FFmpeg [4] and again using Kdenlive [5]. I'll quickly show you how to prepare the transition animation and then cut to the business of building the transition in Natron.

So, first things first, you'll need to get your animation ready. Step one is to split the animation clip into its individual frames:

ffmpeg -i runner.mp4 -ss 00:02:09 -t 00:00:07 runner_frame%04d.png

Broken down, this instruction is made up of the following components:

• runner.mp4 is the video from which you are going to extract the frames.
• -ss 00:02:09 is the starting point of the sequence of frames you want to extract – in this case, two minutes and nine seconds into the clip.
• -t 00:00:07 is the length of the sequence in frames – in this case, seven seconds.
• runner_frame%04d.png gives each frame a sequential name, like runner_frame0000.png, runner_frame0001.png, runner_frame0002.png, runner_frame0003.png, etc.

Next you need to make the green background transparent and cut out the running man. The ImageMagick command

convert runner_frame0000.png -fuzz 40% -fill none -opaque "#13ff09" -trim alpha_cropped_runner_frame0000.png

will do both on your first frame, so that Listing 1 will do it on all the frames.

for i in runner_frame0*.png;\
do convert $i -fuzz 40% -fill none -opaque "#13ff09" -trim alpha_cropped_${i};\
done

Get rid of (or move) the original clip and the original runner_frame*.png frames, leaving only the cut out frames with transparent backgrounds in their own directory. Next, you can use the customtransitions.sh script you saw in [5] to finish off the preparation of the animation frames. To save time, you can download the script [6].

Figure 1 shows the desired result.

Figure 1: You want to process the frames from the original animation (1) so that they end up looking like (3).

Natronizing

Time to fire up Natron.

A new blank Natron project contains only a Viewer node. The first thing you're going to do is right-click in an empty area in the Node Graph (lower left-hand pane) and go to Image. Pick a Read node. This will open a file explorer. Navigate to where you have saved your first clip in the transition sequence and pick that. The Read node will automatically connect to the viewer, as shown in Figure 2. The first frame from your clip will show up in the preview window.

Figure 2: Your clip connected to the Viewer node.

Next bring in the animation frames you created with customtransitions.sh. To do that, create a new Read node and, when the file explorer appears, choose Sequence from the dialog's bottom left-hand corner. Navigate to where customtransitions.sh stored your processed frames and choose the chroma###.png series of frames.

If Natron connects the sequence to the Viewer, disconnect it by clicking and dragging on the arrow connecting the two nodes.

To composite the two inputs, clip 1 and your animation sequence, you need a Merge node, so right-click in the Node Graph and choose Merge | Merge from the pop-up menu. Select the new Merge node and, in its property box (column on the left of Natron's window), change the Operation value to color (selecting this option from the drop-down list).

Connect Merge's A arrow to your video clip and its B arrow to the animation sequence, as shown in Figure 3.

Figure 3: Merging the animation sequence onto clip 1.

The preview pane will show something like what you see in Figure 4, which is a bit more than halfway through the transition.

Figure 4: Animation and clip 1 merged and a bit more than halfway through the transition.

It is time to bring in clip 2. In the same way as before, create a Read node and navigate to the where you have stored the second clip to which you want to transition. If Natron connects it to your tree of nodes, disconnect it and bring in a ChromaKeyer node by right-clicking in the Node Graph and choosing Keyer | ChromaKeyer from the menu.

Connect the output from your Merge node into the ChromaKeyer node and your clip 2 node to the ChromaKeyer's Bg arrow, as shown in Figure 5.

Figure 5: Connecting your second clip to finish off the transition.

The final step is to tell the ChromaKeyer node what color to turn transparent. In the node's properties box, you can either adjust the Key color RGB values by hand, or, if you click in the color box (by default set to black), a little color picker will appear. Move it over to the preview pane, hold down Ctrl, and click on the green section of your animation clip. Presto! Clip 2 now shines through where before was garish green (Figure 6).

Figure 6: The finished product shows part of clip 1, the animated transition dude, and part of clip 2.

Time Offsetting

Your transition is all but done, but so far you have only seen how to apply it to the beginning of the first clip. As you are engineering a transition from clip 1 to clip 2, this is not what you would usually want to do. On the contrary, what you really want to do is place the animation at the end of clip 1. You also want to push the beginning of clip 2 towards the end of clip 1, to when the transition starts.

Although this is bordering on video-editing territory (which is not Natron's purpose), in this case at least, it is legitimate thing to do. To move the beginning of an effect or a clip along the timeline, you can use a TimeOffset node (right-click, and then Time | TimeOffset). Hook it up as shown in Figure 7 and, in the TimeOffset node's property box, change the Time Offset (Frames) value to, say, 100 to make the animation transition start on frame 100 of the overall project.

Figure 7: The TimeOffset node allows you to push effects back in the timeline.

You can now select and right-click on the TimeOffset node and clone the node by choosing Edit | Clone Nodes from the pop-up menu. Use the cloned node (a pinkish red in Figure 7) to delay the start of clip 2 as well, so it coincides with the beginning of the animation. By cloning the node (instead of just creating a new one), you can guarantee that both clips will start at the same point in the project.

Although we have made sure all our clips are the same size for this experiment, you can also resize a clip on the fly with the Resize node (right-click, and then Transform | Resize). However, if you dragged in a large clip, the view port may still show a great big empty area (the size of the biggest clip) even after you have resized it. This is normal: Natron by default resizes your project to the size of your largest clip's frame. To fix this, you need to readjust the project's settings. Mouse over an empty space within the node working area and press the S key on your keyboard. The project settings will show up in the Properties column on the right. Change the size of the project in the Output Format field.

Rendering

To render your project, add a Write node (right-click, and then Image | Write) and plug the ChromaKeyer node in to it, as shown in Figure 8.

Figure 8: Use a Write node to render your clip.

Create a new directory and pick a name for your frames. Something like transition####.jpg will work. Thanks to the #### part of the name, Natron can generate names for your files, such as transition0000.jpg, transition0001.jpg, transition0002.jpg, etc.

Although most of the Write node's values are okay, you may want to change the Frame Range to Manual and then set the first and last frame to render. You should also change the Input Premult value to Opaque, or the chroma part of your animation sequence will show up white or black instead of transparent, depending on the output format you choose.

Tracking

Tracking is one of the more interesting and useful things Natron does quite well. A tracker automatically follows a portion of an image from frame to frame in a video clip.

Say you place a tracker on your main character's motorbike. If the tracking is done correctly, you can have Natron follow the motorbike all through a scene. You can use tracking data to stabilize a shot around the object: Although the motorbike is moving in the original clip, in the modified clip using trackers, you could make the whole environment move around the motorbike, giving your clip a dreamy, giddy feel.

How successfully Natron manages to track something will depend on the clip's quality. A high number of frames per second, a high definition, and a high contrast of the object you want to track with regard to the background is what is going to make tracking easier.

Tracking works on the principle that two consecutive frames within the same sequence (i.e., with no cuts between them) will be similar to each other. If you have someone riding the motorbike from left to right in frame 1, frame 2 will show something very similar, with that same someone, on the same motorbike, riding in the same direction. The only difference will be that the rider and their bike will have moved ever so slightly to the right of the frame. A program can track those slight changes, comparing frame 1 to frame 2, and keep a register of what has changed by inputting data into an array. If you wanted to track the position of the motorbike's back wheel, for example, Natron can create an array with the relative x and y position of said wheel throughout the whole sequence.

That, at least, is the theory. In real life, videos are wobbly, making frames blurry, and changes in light often throw the tracker … well, off track. This means the tracker needs some human assistance.

Load the clip you want to track. I used some stock footage of a motorcycle stunt [7] I found on Pixabay. Bring out a Tracker node (right-click, and then Transform | Tracker), and place it between your clip's Read node and the Viewer node.

Move to the first frame in your clip and double-click on the Tracker node to bring up its properties box to the top of the stack on the right. In your Tracker node's properties box, make sure you are on the Tracking tab and locate the largish text box at the bottom. Add a new tracker by clicking on the + at the bottom of text box (Figure 9). A new tracker will appear in the viewport showing the preview. It looks like a square bullseye and appears in the center of the viewport.

Figure 9: The Tracker node property box with two trackers.

By clicking and dragging on the tracker's central point, you can drag the tracker onto the element you want to track. In the motorbike stunt video, I picked the nearest bike's back wheel. I placed the tracker's central point over the wheel's axle. You can push and pull on the tracker's handles so that it roughly covers the shape of the element you want to track (Figure 10).

Figure 10: A tracker following a stunt bike's back wheel.

With the tracker in place, theoretically you could just press the button with the right-pointing blue arrow in the upper left-hand corner of the preview pane and hope for the best. But the tracker is guaranteed to get lost. The correct thing to do is to place keyframes along the timeline so that the tracker only has to track for short stretches of frames.

The clip of the stunt is only 112 frames long, so you can place a keyframe every 10 frames. To do that, select the track you are going to work with by clicking on it in the property box (in Figure 10, the track1 track is selected), and then make sure you are on the first frame of the segment you want to track and that the tracker is covering the element you want to track. Move to frame 10 and move the tracker by hand (click on the tracker's central point and drag it) to the new position of the element. In the case of the motorbike stunt video, you move the tracker to every new position of the motorcycle's wheel.

Proceed through the clip, relocating the tracker every 10 frames. Each time you change the tracker's position, Natron will insert a new keyframe automatically into your timeline. Keyframes show up like little hourglasses in the timeline, and the Track keyframe value in the Tracking property box will have a dark blue background for keyframes and a light blue background for normal frames.

Once you have inserted all the keyframes, go back to the first frame and click on the right-pointing blue arrow button and hopefully the tracker will run from keyframe to keyframe, inserting all intermediate tracking data.

This is unlikely to happen, however. If the tracker gets confused and loses the thing it is tracking, it will stop on the frame that is causing problems. You then have to reposition the tracker by hand (which will insert a new keyframe) and start tracking from that point onward again.

To locate problematic stretches on a track, you may also want to toggle the showError button located over the preview. This button is located third from the right on the tracker toolbar, and its icon looks like a triangle with cut off vertices. Once toggled, Natron will color each point on the track in a different color: Green means the tracker had no problem at that point of the track, Orange indicates that there was some confusion, and Red shows that the tracker is probably completely lost. It is a good idea to scroll back to the red points, readjust the tracker by hand, and start it off again from that point onward.

Stable Genius

Once Natron has tracked until the end of the clip, you can stabilize the video. In the Tracking properties box, click on the Transform tab and choose Stabilize from the Motion Type drop-down list. And that is all there is to it: When you run through the video, the object you were tracking will remain in the same position relative to the sides of the viewing port, while the rest of the frame moves around it.

Things get wild when you track two points, like both wheels on the motorbike. Then the frame doesn't only move up and down but also revolves around the tracking point (Figure 11), as one tracking point revolves around the other. The effect is quite striking, if not a little dizzying.

Figure 11: The stuntmen in this frame are actually flying upside down through the air, but, by tracking the wheels on one of the bikes, it seems like they are static, and it is the rest of the world that is doing somersaults.

You can see an example of tracking and stabilization, revolving frames and all, online [8].

Next Time

Another interesting use of tracking is to use it to seamlessly insert elements into footage that were not there originally, like posters on walls or app-screen mockups onto phones. This requires careful tracking, though, and a few more techniques. Have fun!