Understanding Rotoscoping for VFX Artists

Rotoscoping is a vital process within the visual effects pipeline. Whether you need to take out elements from live-action footage or bring elements in, you need to have a strong understanding of this very important technique. This article will give you an understanding of rotoscoping and its use, as well as some helpful tips when rotoscoping for your next piece of footage. 

What Is Rotoscoping?

Rotoscoping is one of the oldest animation techniques still in use. In the early days of cinema, rotoscoping meant tracing over live action footage frame by frame for use in animated films. That way the animator could get very realistic movements by tracing over the actor in the scene. Think of it as the motion capture we use today.

What Is Rotoscoping in VFX?

In visual effects, the process, although similar, has a different purpose. Rotoscoping for VFX is used to create a matte or mask for an element, so it can be extracted out to place on a different background, masked out so colors can be changed, or any other set of reasons. It is more widely used than many realize. Rotoscoping gives filmmakers opportunities to produce scenes that would otherwise be difficult, expensive, dangerous, or impossible to film in real life. 

How Does VFX Rotoscoping Work?

The rotoscoping artist (or roto artist for short) traces an object using a set of tools within the compositing software to create a new alpha channel for a specific part of an image sequence or video. Unlike computer generated imagery that can easily add an alpha channel to its images, footage taken directly from a camera has no alpha data. 

Therefore, a roto artist will need to manually create that alpha by tracing over the elements within the video. A rotoscoping artist creates different shapes around an object and animates those shapes to match the movement on each frame.

.

Depending on the complexity of the shot, the process of rotoscoping can take hours or even days to complete. The use of blue and green screens can make the process of compositing different elements into a scene much easier, but not every shot can take advantage of blue or green screens, so rotoscoping still plays a large role in the production of visual effects. Because rotoscoping is used in just about every movie and television show that utilizes visual effects, it's no surprise that roto artists are a vital role within the VFX pipeline. 

There are many different compositing applications with the tools needed for rotoscoping, aided at times by motion-tracking and onion-skinning software. Rotoscoping is an art form in itself, so mastering it will not be something done overnight. The following tutorials will give you a strong foundation in creating good rotoscoping data: 

• Your First Day of Rotoscoping in NUKE

• Introduction to Rotoscoping in After Effects

Rotoscoping Tips

Keep Control Points to a Minimum

When you begin to learn rotoscoping, your first instinct might be to create as many points as possible in order to properly outline the element within the footage. However, it is better to use a minimal amount of points and only what is needed to trace the subject. If a matte's outline changes over several frames, it can have a jittery look when played back, and it's much harder to keep track of too many points. 

A good technique is to find the most complex shape that the subject is in, and outline that first so you know the most points you'll ever need for that rotoscope. The rotoscoping tools in your compositing software have very powerful curve manipulation abilities that can create complex curved lines with a minimal amount of points.

Create Separate Shapes

If the object or person is a very complicated shape, do not try to use one shape for the entire subject. It is best to separate different limbs, fingers, etc. into different sub-shapes. 

For instance, parts that are going to be moved independently of each other should be separated, like the hand, the fingers, the forearm, and the arm. If you tried to follow the outline of a complex movement, like two characters fighting, it would be nearly impossible to follow the actions accurately with one single outline.

Use as Few Keyframes as Possible 

When you are animating the rotoscope to follow along with the movement of the element in the footage, you typically don't need to create a new keyframe for every frame. The computer will automatically interpolate between keyframes and will generally produce the desired results. 

In the same way that an animator would first draw in the key poses, you can find the frames where the most movement occurs and adjust the shapes accordingly. If the outline is not following perfectly with the subject, you can then go in and add another keyframe to make that adjustment.

Study the Footage 

Before you start rotoscoping, you should study the footage you will be working with. Are there large changes in direction? Do parts of the subject get obscured by other elements within the scene? Are there big camera shakes that could be stabilized to make the rotoscoping process easier for you? By knowing the footage, you can better determine the timeframe for the project, as well as the best way to approach the rotoscoping.

Consider Animation Principles 

Rotoscoping is a lot like animation; you are creating keyframes and animating the outline of the subject within the footage. Just as an animator incorporates techniques such as arcs (the visual path of an object or action) and ease in and ease out (the idea that movement needs time to accelerate and slow down) when working on a project, you should as well. 

After all, animation principles were based off of real-world studies, and you'll see these principles in action while tracing your footage. Thinking about these things while you are rotoscoping VFX will help you figure out early on how subjects are likely to move and where the ease ins and ease outs would occur in the footage so you can place the keyframes in the right spot. 

Though rotoscoping may seem tedious and not that exciting, it’s an important technique for any VFX artist to know. Even if you are more of an effects person who likes to create the explosions, you should still know this technique and understand the importance of it within the visual effects world. So the next time you're tasked with rotoscoping, try implementing some of these tips into your projects. 

Learn more with VFX compositing tutorials in our training library and VFX compositing articles on our blog.