Digital Compositing for Film and Video Production Workflows and Techniques

_{Chapter 1 - Getting Started}

_{1.1 About the 5th Edition}

_{1.2 How this Edition is Organized}

_{1.2.1 Part 1 - The Basic Comp}

_{1.2.2 Part 2 - Beyond the Basics}

_{1.3 Web Content}

_{1.4 What’s New in the 5th Edition}

_{1.4.1 Major New Topics}

_{1.4.2 New Content}

_{1.5 Gold Mines}

_{1.6 Retired Material}

_{1.7 Tool Conventions}

_{1.7.1 The Slice Tools}

_{1.7.2 Flowgraphs}

_{1.7.3 Color Look up Tables (LUTs)}

_{1.8 Data conventions}

_{1.8.1 Floating Point Data}

_{1.8.1.1 Banding} 

_{1.8.1.2 Clipping}

_{1.8.2 Linear Ligt Space}

_{1.8.3 HDR Images}

_{1.8.4 Stops}

_{Chapter 2 - Pulling Keys}

_{2.1 Lumakeys}

_{2.1.1 How Lumakeys Work}

_{2.1.2 Making Your Own Luminance Image}

_{2.1.2.1 Variations on the Luminance Equations}

_{2.1.2.2 Non-luminance Monochrome Images}

_{2.1.3 Making Your Own Lumakeyer}

_{2.2 Chromakeys}

_{2.2.1 How Chroma Keys Work}

_{2.2.2 Making Your Own Chroma Keyer}

_{2.2.3 Making a 3D Chroma Keyer}

_{2.3 Difference Mattes}

_{2.3.1 How Difference Mattes Work}

_{2.3.2 Making Your Own Difference Matte}

_{2.3.2.1 Making the Difference Image}

_{2.3.2.2 Making the Difference Matte}

_{2.4 High Bypass Filters}

_{2.4.1 Creating the Filter}

_{2.4.2 Digital Airbrushing}

_{2.5 Color Difference Keys}

_{2.6 RGB Edge Extension Technique} 

_{2.7 Rotoscoping}

_{2.7.1 Overarching Strategy}

_{2.7.2 Control Point Coherency}

_{2.7.3 Shape Breakdown}

_{2.7.3.1 Hierarchical Articulation}

_{2.7.3.2 Organization}

_{2.7.4 Bezier or B-spline?}

_{2.7.5 Keyframe Strategies}

_{2.7.5.1 On 2’s}

_{2.7.5.2 Binary Multiples}

_{2.7.5.3 Bifurcation}

_{2.7.5.4 Motion Extremes}

_{2.7.6 Motion Blur}

_{2.6.1 Spline Placement}

_{2.7.6.2 Edge Decontamination}

_{2.7.7 Inspection}

_{Chapter 3 - Keyers}

_{3.1 How Keyers Work}

_{3.1.1 Calculating the Color Difference Matte}

_{3.1.1.1 The Theory}

_{3.1.1.2 Pulling the Raw Matte}

_{3.1.1.3 A Simplified Example}

_{3.1.1.4 A Slightly More Realistic Case}

_{3.1.1.5 And Now, the Real World}

_{3.1.2 Matte Edge Penetration}

_{3.1.3 Scaling the Raw atte}

_{3.2 The After Effects Keyer}

_{3.2.1 Step-by-Step Procedure}

_{3.2.2 Flowgraph of the After Effects Keyer}

_{3.3 Typical Greenscreen Problems}

_{3.3.1 Over Exposed}

_{3.3.2 Under Exposed}

_{3.3.3 Impure Greenscreens}

_{3.3.4 Uneven Lighting}

_{3.4 Preprocessing the Greenscreen}

_{3.4.1 Denoise and Degrain}

_{3.4.2 Local Suppression} 

_{3.4.3 Channel Clamping}

_{3.4.4 Screen Correction}

_{3.4.4.1 Fine Hair Detail}

_{3.4.4.2 Creating the Clean Plate}

_{3.4.4.3 Screen Correction Procedure}

_{3.4.4.4 Pictographic Flow Chart}

_{3.4.4.5 Flowgraph of the SCreen Correction PRocedure 3.4.4.6 Nuke IBK Procedure}

_{Chapter 4 - Refining Mattes}

_{4.1 Gamma Slamming}

_{4.2 Garbage Mattes}

_{4.2.1 Pre-matting}

_{4.2.2 Post-matting}

_{4.3 Filtering the Matte}

_{4.3.1 Noise Suppression with a Median Filter}

_{4.3.2 Softer Edges}

_{4.3.3 Controlling the Blur Operation}

_{4.3.3.1 The Blur Radius}

_{4.3.3.2 The Blur Percentage}

_{4.3.3.3 Masking the Blur}

_{4.4  Adjusting the Matte Size}

_{4.4.1 Eroding a Matte with Blur and Scale}

_{4.4.2 Dilating a Matte with Blur and Scale}

_{4.4.3 Blurring Out}

_{4.4.4 Sculpting Edges}

_{4.5 Edge Masks}

_{4.6 Merging Mattes}

_{Chapter 5 - Spill Suppression}

_{5.1 Sources of Spill}

_{5.2 The Despill Operation}

_{5.3 Despill Algorithms}

_{5.3.1 Green Limited by Red}

_{5.3.2 Implementing the Algorithm}

_{5.3.3 The Spillmap}

_{5.3.4 Green Limited by the Average of Red and Blue}

_{5.3.5 An Adjustable Despill}

_{5.3.6 What About Blue Spill?}

_{5.3.7 Refining the Despill}

_{5.4 The Unspill Operation}

_{5.4.1 How to Set It Up}

_{5.4.2 Grading to the Backing Color}

_{5.5 Despill Artifacts}

_{5.5.1 Finding the Artifacts}

_{5.5.2 Hue Shifts}

_{5.5.3 Dark Edges}

_{5.5.4 Fixing Despill Artifacts}

_{5.6 Edge Grading}

_{5.7 Edge Extension}

_{Chapter 6 - Image Blending}

_{6.1 Image Blending in Linear Light Space}

_{6.1.1 Image Blending Operations}

_{6.1.2 Compositing Operations}

_{6.1.3 Matching the Look of sRGB in Linear}

_{6.1.3.1 All sRGB Color Space}

_{6.1.3.2 sRGB Within Linear}

_{6.2 Alpha Compositing Operations}

_{6.3 Image Blending Operations}

_{6.3.1 The Screen Operation} 

_{6.3.1.1 Adjusting the Appearance}

_{6.3.2 The Weighted Screen Operation}

_{6.3.3 Multiply}

_{6.3.3.1 Adjusting the Appearance}

_{6.3.4 Maximum}

_{6.3.5 Minimum}

_{6.3.6 Absolute Difference 6.4 Adobe Photoshop blending modes} 

_{6.4.1 Simple Blending Modes}

_{6.4.2 Complex Blending Modes}

_{6.5 Slot Gags}

_{6.6 Retiming Clips}

_{6.6.1 Constant Speed Changes}

_{6.6.2 Variable speed changes}

_{6.6.3 Interpolation methods}

_{6.6.3.1 Nearest neighbor}

_{6.6.3.2 Frame average}

_{6.6.3.3 Motion estimation}

_{Chapter 7 – The Composite}

_{7.1 Premultiply vs. Unpremultiply}

_{7.1.1 Premultiply}

_{7.1.2 Unpremultiply}

_{7.1.3 Not Premultiplied}

_{7.1.4 The Double Premultiply}

_{7.2 Compositing Operations}

_{7.2.1 The Over Composite} 

_{7.2.2 The KeyMix Composite}

_{7.2.3 The AddMix Composite}

_{7.2.3.1 How it Works}

_{7.2.3.2 How to Build It}

_{7.2.3.3 How to Use It}

_{7.2.4 The Processed Foreground Method}

_{7.2.4.1 The Workflow}

_{7.2.4.2 What to Watch Out For 7.3 Compositing Inside the Keyer}

_{7.3.1 Adding a Holdout Mask}

_{7.3.2 Adding a Garbage Matte}

_{7.3.3 Going Off Screen}

_{7.3.4 Grading and Grain Management}

_{7.4 Compositing Outside the Keyer}

_{7.4.1 The Basic Workflow}

_{7.4.2 The Uberkey}

_{7.4.3 Soft Key/Hard Key 7.5 Virtual Production}

_{7.5.1 How Virtual Production Works}

_{7.5.2 The Effects on Compositing}

_{Chapter 08 – Compositing CG}

_{8.1 Multi-Pass CGI Compositing}

_{8.1.1 Lighting Passes}

_{8.1.1.1 Revising the Beauty Pass}

_{8.1.2 AOVs}

_{8.1.3 Normals Relighting}

_{8.1.4 ID Passes}

_{8.2 Cryptomattes}

_{8.2.1 How Cryptomattes Work}

_{8.2.2 How Cryptomattes are Used}

_{8.3 EXR File Format}

_{8.3.1 Film Scans}

_{8.3.2 Linear Images}

_{8.3.3 Arbitrary Image Channels}

_{8.4 HDR Images}

_{8.5 Deep Compositing}

_{8.5.1 Deep Images}

_{8.5.2 The Layering Complexity Problem}

_{8.5.3 The Depth Compositing Edge Problem}

_{8.5.4 The Re-rendering Problem}

_{8.5.5 Deep Compositing with Live Action}

_{Chapter 9 – Tracking and Transformations}

 _{9.1 Geometric transforms}

_{9.1.1 2D Transforms}

_{9.1.1.1 Translation}

_{9.1.1.2 Rotation}

_{9.1.1.3 Resize vs. Scale}

_{9.1.1.4 Skew}

_{9.1.1.5 Corner Pinning}

_{9.1.2 Managing Motion blur}

_{9.1.2.1 Transform motion blur}

_{9.1.2.2 Motion UV motion blur}

_{9.1.2.3 Speed changes}

_{9.1.3 3D Transforms}

_{9.1.4 Filtering}

_{9.1.4.1 The Effects of Filtering}

_{9.1.4.2 Twinkling Starfields}

_{9.1.4.3 Choosing a Filter}

_{9.1.5 Image Alignment}

_{9.1.5.1 Embossed Alignment}

_{9.1.5.2 Edge Detection Alignment}

_{9.1.5.3 Pivot Point Alignment}

_{9.2 Image Displacement}

_{9.3 Warps and Morphs}

_{9.3.1 Mesh Warps}

_{9.3.2 Spline Warps}

_{9.3.3 Morphs}

_{9.3.4 Tips, Tricks and Techniques}

_{9.4 Point Tracking}

_{9.4.1 The Tracking Operation}

_{9.4.1.1 Selecting Good Tracking Targets}

_{9.4.1.2 Bad Tracking Targets}

_{9.4.1.3 Tracker Enable/Disable}

_{9.4.1.4 Offset Tracking}

_{9.4.1.5 Keep Shape and Follow Shape}

_{9.4.1.6 Pre-processing the Clip}

_{9.4.1.7 Coping with Grain}

_{9.4.1.8 Tracking Workflow}

_{9.4.1.9 Cleaning up Tracking Data}

_{9.4.1.10 The Stability Test}

_{9.4.1.11 Reasons for Failure}

_{9.4.2 Matchmove}

_{9.4.2.1 2D Transforms}

_{9.4.2.2 The Reference Frame}

_{9.4.2.3 Corner Pinning}

_{9.4.3 Stabilizing}

_{9.4.3.1 The Repo Problem}

_{9.4.3.2 Motion Smoothing}

_{9.4.3.3 Stabilizing For Rotoscoping}

_{9.5 Planar Tracking}

_{9.5.1 The Planar Grid}

_{9.5.2 Drift Correction}

_{9.5.3 Exporting Data}

_{9.5.4 Roto Assist}

_{Chapter 10 – 3D Compositing}

 _{10.1 A Short Course in 3D}

_{10.1.1 The 3D Coordinate System}

_{10.1.2 Vertices}

_{10.1.3 Meshes}

_{10.1.4 Surface Normals}

_{10.1.5 UV Coordinates}

_{10.1.6 Map Projection}

_{10.1.7 UV Projection}

_{10.1.8 3D Geometry}

_{10.1.9 Geometric Transformations}

_{10.1.10 Geometric Deformations}

_{10.1.10.1 Image Displacement}

_{10.1.10.2 Deformation Lattice}

_{10.1.11 Point Clouds}

_{10.1.12 Lights}

_{10.1.13 Cameras}

_{10.1.14 Shaders}

_{10.1.15 Reflection Mapping}

_{10.1.16 3D Environment Maps}

_{10.1.17 Ray Tracing}

_{10.1.18 Image Based Lighting}

_{10.2 3D Compositing}

_{10.2.1 Camera Tracking}

_{10.2.1.1 Step 1 - Tracking}

_{10.2.1.2 Step 2 - The Solve}

_{10.2.1.3 Step 3 – Build the Scene}

_{10.2.1.4 Placing the Geometry}

_{10.2.1.5 A Large Outdoor Scene}

_{10.2.1.6 Monitor Insert Shots}

_{10.2.2  Camera Projection}

_{10.2.3  Multiplane Shots}

_{10.2.4 Set Extension}

_{10.2.4.1 Step 1 – Camera Tracking}

_{10.2.4.2 Step 2 – Preps for the DMP}

_{10.2.4.3 Step 3 – Creating the Digital Matte Painting}

_{10.2.4.4 Step 4 – The DMP Render}

_{10.2.4.5 Compositing the DMP}

_{10.2.5 Pan and Tile}

_{10.2.6 3D Backgrounds}

_{10.2.7 3D compositing from 2D images}

_{10.2.8 Alembic Geometry}

_{10.2.8.1 The Simple Case}

_{10.2.8.2 Scenegraphs}

_{10.2.8.3 Advantages Over FBX}

_{10.2.9 Universal Scene Description (USD)}

_{10.2.9.1 The Scenegraph}

_{10.2.9.2 A Case Study}

_{Chapter 11 – Digital Images} 

_{11.1 Color Space} 

_{11.1.1 Color Models}

_{11.1.2 Primary Chromaticities}

_{11.1.3 Gamut}

_{11.1.4 The Transfer Function}

_{11.1.5 White Point}

_{11.2 HDTV}

_{11.2.1 Rec. 709 Color Space}

_{11.2.2 Frame Formats}

_{11.2.3 Scan modes}

_{11.2.4 Working with interlaced video}

_{11.2.5 De-interlacing}

_{11.2.5.1 Field Averaging}

_{11.2.5.2 Optical Flow Interpolation}

_{11.2.6 Color Subsampling}

_{11.2.7 Keying 4:2:2 Video}

_{11.2.8 Frame rates}

_{11.2.8.1 Integer:  24, 25, 30, 60 fps}

_{11.2.8.2 Float:  23.98, 29.97, 59.94 fps}

_{11.2.9 Timecodes}

_{11.2.10 Video File Formats}

_{11.2.11 Film Scanners}

_{11.2.12 Anamorphic video}

_{11.3 UHDTV}

_{11.3.1 Rec. 2020 Color Space}

_{11.4 Digital Cinema images}

_{1.14.1 P3 Color Space}

_{11.4.2 Digital Cameras vs. Film}

_{11.4.3 The Bayer Array}

_{11.4.4 Sensor Crop}

_{11.4.5 HFR – High Frame Rate}

_{11.4.6 The DCI}

_{11.5 Working in linear}

_{11.5.1 What exactly is linear?}

_{11.5.2 Color Correcting}

_{11.5.3 Transformations and Filtering}

_{11.5.4 CG}

_{11.6 Log Images}

_{11.6.1 What are Log Images?}

_{11.6.2 Why we need log images}

_{11.6.2.1 Data Compression}

_{11.6.3 Working with Log Images}

_{11.7 Metadata}

_{11.8 Artificial Intelligence and VFX}

_{11.8.1 A Brief History}

_{11.8.2 Terminology}

_{11.8.3 AI for Compositing}

_{11.8.3.1 Rotoscoping}

_{11.8.3.2 Speed Changes}

_{11.8.3.3 Upscale and Sharpen}

_{11.8.3.4 Inpainting}

_{11.8.3.5 Greenscreen Keying}

_{11.8.3.6 Image Generation}

_{11.8.3.7 De-aging with Deep Fakes}

_{11.8.3.8 Face Replacements}

_{11.8.4 The Future}

_{11.9 OpenColorIO}

_{11.10 ACES Color Management}

_{11.10.1 The ACES Workflow}

_{11.10.2 The ACES Gamut}

_{11.10.3 What About Video Productions?}

 _{Chapter 12 – Color Correction}

_{12.1 The Behavior of Light}

_{12.1.1 The Inverse Square Law}

_{12.1.1 Diffuse Reflections}

_{12.1.2 Specular Reflections}

_{12.1.3 Bounce Light}

_{12.1.4 Scattering}

_{12.1.5 Polarized Light}

_{12.1.6 The Fresnel Effect}

_{12.1.7 Spectral Emissions}

_{12.2   Gamma}

_{12.3   The Effect of Color Operations}

_{12.3.1 Lift}

_{12.3.2 Gamma}

_{12.3.3 Gain}

_{12.3.4 Offset}

_{12.3.5 Saturation}

_{12.3.6 Color Grading vs. Color Correcting}

_{12.3.7 Increasing Contrast with the “S” Curve}

_{12.3.8 Histograms}

_{12.3.9 Channel Swapping}

_{12.3.10 Premultiply vs. Unpremultiply}

_{12.4  Matching the Light Space}

_{12.4.1 Brightness and Contras}

_{12.4.1.1 Matching the Black and White Points}

_{12.4.1.2 Matching the Midtones}

_{12.4.1.3 Gamma Slamming}

_{12.4.2 Matching Color}

_{12.4.2.1 Grayscale Balancing}

_{12.4.2.2 Flesh Tones}

_{12.4.2.3 The “Constant Green” Method of Color Correction}

_{12.4.2.4 Daylight}

_{12.4.2.5 Specular Highlights}

_{12.4.3  Lighting Direction}

_{12.4.4  Quality of Light Sources}

_{12.4.4.1 Creating Softer Lighting}

_{12.4.4.2 Creating Harsher Lighting}

_{12.4.5  Non-linear Gradients for Color Correction}

_{12.4.6  Colorized Gradients}

_{12.4.7  The DI Process}

_{12.4.8  A Checklist}

_{Chapter 13 – Camera Effects} 

_{13.1 Lens Effects 1}

_{3.1.1 Lens Distortion}

_{13.1.2 Depth of Field}

_{13.1.3 Vignetting}

_{13.1.4 Lens Defects}

_{13.1.4.1 Spherical Aberration}

_{13.1.4.2 Astigmatism}

_{13.1.4.3 Chromatic Aberration}

_{13.1.5 Glows and Flares}

_{13.1.5.1 Lens flare}

_{13.1.5.2 Lens Filter Flare}

_{13.1.5.3 Diffraction Glows}

_{13.1.5.4 Veiling Glare}

_{13.1.6 Grain}

_{13.2  Lens Distortion Workflows} 

   _{13.2.1 CG Over Live Action}

_{13.2.2 Live Action Over CG}

_{13.2.3 CG Over CG}

_{13.2.4 Live Action Over Live Action}

_{13.3  Matching the Focus}

_{13.3.1 Using a Blur for Defocus}

_{13.3.2 How to Simulate a Defocus}

_{13.3.3 Sharpening}

_{13.3.3.1 Sharpening Operations}

_{13.3.3.2 Unsharp Masks}

_{13.3.3.3 Making Your Own Unsharp Mask}

_{13.4 Rolling shutter}

_{Chapter 14 – Sweetening the Comp} 

_{14.1 Layer Integration}

_{14.2 Interactive Lighting}

_{14.3 Light Wrap}

_{14.4 Edge Blending}

_{14.5 Creating Shadows}

_{14.5.1 Edge Characteristics}

_{14.5.2 Density}

_{14.5.3 Color}

_{14.5.4 Faux Shadows}

_{14.5.4.1 Caveats}

_{14.5.4.2 The Shadow Mask}

_{14.5.5 Shadow Warping}

_{14.5.6 Contact Shadows}

_{14.6 Atmospheric Haze}

_{14.7 Adding a Glow}

_{14.8 Grain Management}

_{14.8.1 Grain Characteristics}

_{14.8.2 Regraining Techniques}

_{14.8.2.1 Grain Check Procedure}

_{14.8.2.2 Generated Grain}

_{14.8.2.3 Extracted Grain}

_{14.8.3 Workflows}

_{14.8.3.1 Live Action Over Live Action}

_{14.8.3.2 Live Action Over CG}

_{14.8.3.3 CG Over Live Action}

_{14.8.3.4 CG Over CG}

_{14.8.3.5 Still Photos}

_{14.9 Clipping}

 

Biography

Steve Wright is a visual effects pioneer and a 20+-year veteran of compositing on over 80 feature films and many broadcast television commercials. With extensive production experience and a knack for the math and science of visual effects, he is a world-renowned expert on visual effects compositing. Since 2005 he has been a master trainer in compositing visual effects, providing staff training to over 35 visual effects studios worldwide, including Pixar Animation Studios, Disney Feature Animation, Troublemaker Studios, New Deal Studios, Reliance MediaWorks and many others, and has trained well over 2000 artists in compositing.

Visit Steve’s training website at www.fxecademy.com

"I have been a lecturer in Film and Visual Effects for more than 10 years and Steve Wright’s Digital Compositing for Film and Video has always been an integral part of my teaching material.

Complex concepts, industry-standard workflow methodologies, and compositing strategies are articulately explained to be easily understood by both Lecturers and students.

Successive editions ensure information is always up to date with workflows integrating new technologies and methods and should be compulsory reading for everyone in the world of Visual Effects."

Shaun Peyman, Senior Lecturer VFX, Media Design School, Auckland, New Zealand

*************************************************************************************************

"Once again, Steve's 5^th Edition of Digital Compositing for Film and Video (the legendary 'blue bible') provides an updated 'must have'. Any Compositor who reads and understands the concepts in this book will not only be set for a fulfilling career, but they will also be leagues ahead of those who don’t."

Scott Chambers, Founder and VFX Supervisor, 808 Ltd., Wellington, New Zealand

**************************************************************************************************

“In a field where technology and creativity meet, this new edition continues to stand as a must-read. Its comprehensive exploration of the art of digital compositing is both enlightening and empowering, offering an agnostic and practical understanding of the craft. Essential for any serious practitioner in the industry.”

Armando Ricalde, Chief Technology Officer, Boxel Studio, Mexico

**************************************************************************************************

"Steve’s 5th Edition is, as before, the best mix out of his hands-on training and production experience, which supports any VFX artist with the essential knowledge needed for the production pipeline. The main advantage of this edition is the expansion of new technologies and methods, which makes his book again an essential must-have for any serious VFX artist."

Dr. Sassi, Senior Trainer, VES, Los Angeles, Maxon-One