Understanding the Importance of Matchmoving for Integrating CG Elements into Live-Action Footage
Mar 10, 2014 • 3 Minute Read
Matchmoving, which is also referred to as motion tracking, is important for any VFX artist to understand. Without it there would be no way to incorporate 3D data into live action footage. With the advancements in software and the machines that run them, matchmoving has become more affordable and faster to accomplish. This article will help you have a stronger understanding of matchmoving and why it's important for you as a VFX artist.
VFX is basically the integration of 3D elements into real-world footage. For instance, Hulk in the Avengers or Optimus Prime in Transformers. To make this accurate and appear as if these 3D models are living alongside the actors, you need a virtual camera that moves exactly like the camera in the live action footage.
This movement ensure anything that gets rendered out in 3D will be from the same angles as the live action footage at all times. This process of matching the movement of the live action footage with a virtual camera is called matchmoving, because you are trying to match the motions of the camera.
There are many different applications with matchmoving capabilities. These programs use a tracking algorithm to lock onto and follow through multiple frames the specific points of real-world footage that an artist has identified. Typically these areas are chosen by the amount of contrast between different areas.
This contrast makes it much easier for the motion tracking algorithm to create accurate tracking information. When an area is tracked, it becomes two dimensional coordinates. These coordinates represent the position of the object or feature across multiple frames.
The use of matchmoving spans across many mediums, including movies and television. You can even see it being used during live broadcasts. For instance, the first down line in football that the viewer sees on their television is a computer generated line which is not actually present on the field.
There are two different forms of matchmoving, 2D and 3D. Two-dimensional matchmoving only tracks features in 2D space and doesn't factor in camera movement or distortion. This can be used for replacing the image on a television or changing the advertisement on a billboard in the live action footage.
Three-dimensional matchmoving makes it possible to interpolate 3D information from 2D footage. You can then use the information generated and a 3D animation application to create a virtual camera.
With the major advancements in technology, many production studios are utilizing something called real-time matchmoving, which is used heavily in the film industry. It allows the director to get a real-time representation of what the final composite will look like; this includes the 3D elements that are going to be included in post production.
Of course, this representation isn't completely accurate but it gives the director and actors a glimpse of what it is going to look like. Often times, actors are working in a greenscreen studio, and this glimpse allows them to actually see set extensions or CG characters right after they do a take. For instance, when Bumblebee in Transformers was interacting with the actors they had to visualize where Bumblebee was going to be standing in the final composite.
Now that you have an understanding of matchmoving and the important role it plays, check out some of these great VFX tutorials to learn how to integrate different elements into your footage, so you can become an even more productive and skilled VFX artist: Your First Day of Tracking in NUKE, Tracking in After Effects, Getting Started with mocha for After Effects.
The use of matchmoving spans across many mediums, including movies and television. You can even see it being used during live broadcasts. For instance, the first down line in football that the viewer sees on their television is a computer generated line which is not actually present on the field.
There are two different forms of matchmoving, 2D and 3D. Two-dimensional matchmoving only tracks features in 2D space and doesn't factor in camera movement or distortion. This can be used for replacing the image on a television or changing the advertisement on a billboard in the live action footage.
Three-dimensional matchmoving makes it possible to interpolate 3D information from 2D footage. You can then use the information generated and a 3D animation application to create a virtual camera.
With the major advancements in technology, many production studios are utilizing something called real-time matchmoving, which is used heavily in the film industry. It allows the director to get a real-time representation of what the final composite will look like; this includes the 3D elements that are going to be included in post production.
Of course, this representation isn't completely accurate but it gives the director and actors a glimpse of what it is going to look like. Often times, actors are working in a greenscreen studio, and this glimpse allows them to actually see set extensions or CG characters right after they do a take. For instance, when Bumblebee in Transformers was interacting with the actors they had to visualize where Bumblebee was going to be standing in the final composite.
Software
You have several different software packages to choose from in order to create great matchmoving data. Mocha is a stand alone motion tracking software that allows artist to quickly solve difficult tracks and export all the data into some of the industry's top applications, like After Effects, NUKE and more. Of course, these applications also have matchmoving capabilities built-in, but sometimes a mixture of different tools and software can save time. If you want to do 3D matchmoving, you will need to ensure your package has those capabilities. Here are a few programs capable of 3D matchmoving: NUKEX, mocha Pro, CameraTracker plug-in for After Effects, and Boujou.Study Your Footage
Before setting your tracking points, you should play back the footage many times so you can get a feel of what's going to happen and the motion of the footage. You also need to look for features that are relatively consistent throughout the entire footage. The things you want to avoid when tracking are heavily shadowed areas or brightly lit areas and in places where the lighting changes on it. This can make it very difficult for the application to figure out what area actually needs to be tracked.Create Tracking Markers
In a more complex matchmoving project, it's often best to incorporate some type of tracking markers. These can be simple markers placed in the footage. It's especially important that these tracking points be non-deformable for object tracking. For instance, creating a CG robotic arm in the place of one of the actor's arms. These tracking features should be placed based on the footage, and they should be hidden by the effect or placed where it will be easy to erase them when compositing. It's also a good idea to have these markers a brighter color so it's easy for the computer to track.
Now that you have an understanding of matchmoving and the important role it plays, check out some of these great VFX tutorials to learn how to integrate different elements into your footage, so you can become an even more productive and skilled VFX artist: Your First Day of Tracking in NUKE, Tracking in After Effects, Getting Started with mocha for After Effects.
