Acknowledgements
Preface
Chapter 1 | A Professional Understanding of Film Visual Effects and Related Subjects
· The origin and historical background of film visual effects |
· Distinguish between visual effects and other similar professional subjects |
· Professional definition of film special effects |
· Professional definition of motion graphic design |
· Understanding effects (FX) assets |
· Understanding traditional post-production |
Chapter 2 | Concepts of Working on Visual Effects Projects and Insights on Recent Trends
· Concept and importance of collaboration in visual effects production |
· Understanding shot scripts for VFX preparation |
· Understanding the use of live-action footage in visual effects production |
· Thoughts on the application of AI in visual effects and creative production industries |
· Thoughts on the influence of virtual production (VP) on visual effects |
Chapter 3 | Interpretation of Department Operations at Top Visual Effects Studios
· An overview of top film visual effects studios |
· Operating platforms: Industry-standard operating platforms for film visual effects |
· Division of departments: Internal division of functions at top VFX studios |
o Art Department & Virtual Art Department
o Animation Department
o Compositing Department
o Crowd Department
o 3D Digital Matte Painting (DMP) / Environment Department
o Data Capture Department
o Editorial Department
o FX Department
o Groom Department
o Imaging Department
o Layout & Previsualization Department
o Lighting Department
o Modeling Department
o Motion Capture Department
o Matchmove Department
o Photography Department
o Production Department
o Rigging Department
o Roto / Prep Department
o Shading and Rendering Department
o Software Research and Development
o Stereoscopic Department
o Texturing Department
· Professional software: Industry-standard software and file formats for film visual effects |
o Operating systems
o Rotoscoping and prep software
o 3D camera tracking software
o Compositing software
o 3D software
o Common file formats used for storing CG assets
o Live-action footage formats used in visual effects
· Examples of the operational framework between departments |
o Key operations conducted in the Editorial Department
o Key operations conducted in the Roto/Prep Department
o Key operations conducted in the Compositing Department
o Key operations conducted in the Lighting Department
o Key operations conducted in the Stereoscopic Department
o Key operations conducted in the Digital Matte Painting (DMP) Department
Chapter 4 | Analysis of the Core Components of Invisible Film Visual Effects
· Understand the color spaces of digital images and the VFX color pipelines |
o Additive color vs. subtractive color
o Scene-referred and display-referred workflows
o Linear space vs. human vision and color perception
o Understanding gamma compensation in digital images
o Understanding linear color workflow in VFX production
o Scene-referred workflow for integrating 3D assets into live-action footage
· Analyses of digital image bit depths and applications in visual effects production |
o Understanding of image bit depths
o The application of image bit depths in visual effects production
· Hands-on example (I): Create a custom LUT with the default Nuke color management |
· ACES color management in VFX projects |
o Essential understanding of ACES
- ACES 2605-1
- ACEScg
- sRGB / Rec709
- DCI-P3
· Hands-on example (II): Create a custom LUT using ACES color management in Nuke |
· The importance of on-site data capture for film visual effects shots |
· Professional understanding of on-set data capture and measurements |
o Total station survey
o Reference photos
- Slate reference photos
- Scene reference photos
- Cameras reference photos
- Lighting reference photos
- Witness cameras
- Total station
- Tracking markers
- Actors/Actresses
- Stunt performers
o Witness camera
o Drift camera
o HDRI shoot
o Roundshot shoot
o Character and prop texture shoot
o Environment and set shoot
o Photogrammetry Shoot
o Capturing reference images for creating 3D models
o Capturing reference images for 3D texturing
o Understand lens distortion and shoot lens grid
o Lens vignetting
Chapter 5 | Visual Effects Project Case Studies
· Introduction to the primary applications used in the case studies |
o The essential understanding of node-based compositing in NukeX
o The essential understanding of 3D applications used in the case studies
· The essential understanding of matte in compositing |
o Approach one: Rotoscoping
o Approach two: CG assets rendered from 3D software
o Approach three: Screen keying
Case Study 1 – Erasing Unwanted Elements from Live-Action Footage |
· Case Study 1: Shot analysis |
o Terminologies mentioned in Case Study 1
o Client requirements
o Camera movements in the original shot
o Production concept
· Analyzing the Nuke compositing script of Case Study 1 |
o Section 1: Original plate
o Section 2: Denoise (BG) plate and 2D tracking
o Section 3: Create clean patches and compositing
o Section 4: Create the foreground matte and final composition
Case Study 2 – Understanding the Importance of 3D Camera Tracking in Live-Action Footage |
· The concept of 3D camera tracking in visual effects shots |
· 3D camera tracking in Case Study 2 |
o The purpose of this case study
o Shot analysis
o Footage information
o Camera information
· Workflow analysis |
o Section 1: Removing lens distortion
o Section 2: 3D camera tracking
o Section 3: Exporting 3D camera tracking data from NukeX to Maya
o Section 4: 3D match-moving test in Autodesk Maya
o Section 5: Testing geometries in NukeX
o Section 6: Restoring original lens distortion
o Section 7: Final output
Case Study 3 – The Library Project |
· Creative ideas and shot analysis |
o Creative ideas
o Shot analysis
· Workflow analysis |
o Part 1 – Creating roto shapes, removing lens distortion from the original plate,
Performing 3D camera tracking, and exporting the tracking data in NukeX
o Part 2 – 3D match-moving, lighting and rendering in Autodesk Maya
o Part 3 – Integrating 3D assets with the original background plate in NukeX
Case study 4 – Sky Replacement |
· Creative brief and shot analysis |
o Creative brief
o Shot analysis
o Purpose of this demonstration
o Nuke script analysis
Section 1 – Asset preparations
- Capture a cloudy sky image and lens grid
- Live-action plate: 3D camera tracking
- Extracting a still reference image to match the camera perspective
- The damaged road surface
Section 2: Compositing starts
- Sky reprojection
- Additive keyer
Section 3: Creating mattes
- Original live-action (BG) plate
- Creating mattes for treetops on the left side of the road
- Creating mattes for the treetops on the right side of the road
- Creating mattes for buildings
- Combining mattes
- Color corrections for the live-action plate
Section 4: Fixing reflections of the original sky on the car roof and motorcycle helmet
- Fixing car roof and helmet reflections
Section 5: Adding light wrap and projecting the damaged road surface
- Implementing a light wrap effect
- Projecting the damaged road surface
- Adding a light beam and lens flare to the motorcycle’s headlight
Case Study 5 – Essential Concepts of Green / Blue Screen Keying and Compositing |
· An essential understanding of green / blue screen keying |
· The importance of chroma subsampling for keying |
· Types of mattes in keying |
o Soft matte
o Hard matte
o Core matte
o Garbage matte
· Footage analysis and Nuke script demonstrations |
o Green screen plate
o Camera information
o Background image
o Nuke script demonstrations
- Creating a (color) Lookup Table (LUT) for Case Study 5
Section 1: Removing film grain from the background image
Section 2: Remove lens distortion from the background image
Section 3: Remove tracking markers from the green screen plate
o Operations in Sections 1 – 3
- Remove film grain before 2D tracking
- 2D tracking
- Remove the tracking marker from frames 1001 to 1241
Section 4: Remove the film grain from the new green screen plate
Section 5: Keying and creating mattes
Section 6: Green screen despill
Section 7: Additive keyer
Section 8: Final output
Case Study 6 – Creating a VFX Shot for the Old Film The Lavender Hill Mob |
· Footage and production analysis |
o Footage information
o Production analysis
o Examples of the required 2D assets
· The overall Nuke scripts in Case Study 6 |
o Nuke script 1: 3D Camera Tracking
- Preparations before 3D camera tracking
- 3D camera tracking
o Nuke script 2: Compositing
- Section 1. Erase shadows on the wall (F160 – F180)
- Section 2. Erase the toy tower held by a pupil (F177 – F183)
- Section 3. Creating mattes for the hands (F100 – F191)
- Section 4. Create clean patches for the wall and suitcase
· Creating the 3D Coke bottle
o 3D match-moving and animation in Maya
o Final rendering and compositing
Case Study 7 – The CG Shoe Project:
Integrating CG Elements Seamlessly with Low-end Live-action Footage |
· Production analysis |
o Footage analysis
o Shot analysis
· The overall workflow across applications |
o 3D camera tracking and matchmoving
o Creating 3D shoes in Maya
- Shoot reference images and create 3D models
- Create texture maps
- Illuminate the scene using an HDR image
o Multi-pass renders
- Beauty pass (primary pass)
- Matte
- Diffuse (no shadow) & Specular (no shadow)
- Indirect
- Shadow
- Reflection
- Occlusion
- Fresnel
- ID pass A & ID pass B
- Shadow on the table
- Motion vector
o Compositing script
- Integration of 3D assets into the background plate
- CG shoe multi-pass compositing
- Color correction
- Create chromatic aberration
- ID pass and color correction
- Copy the matte of the 3D shoe into the main compositing pipeline
- Premultiplied and unpremultiplied images
- Reduce the image quality of the CG shoe
- Add lens distortion to the CG shoe and finalize the composition
Case Study 8 – The Space Capsule Project |
· Determine shooting methods based on the shot script |
o Shooting method
o Shooting location and time
· Shooting live-action footage and capturing on-set data |
o Capturing on-set data and files
- Camera model
- The project frame rate and aspect ratio
- Lens model and focal length
- Filming a lens grid
o Footage color space
o Capture reference images of camera movements
o Capture multiple angles of reference images on location
o Capture High Dynamic Range (HDR) images
o Scan the tunnel to obtain a full-size 3D model
· Live-action footage editorial
· Preparations for 3D camera tracking
· 3D camera tracking for live-action footage
· Matchmoving the background plate in Maya
· Creating the 3D space capsule and animation
· Texturing and lighting
o Creating textures for the 3D space capsule
o Simulating environmental lighting with HDRI
· Multi-pass rendering and final compositing
o Multi-pass renders
o Final compositing script
- Preparations
- Final compositing
· Nuke Scripts – Production-level Examples |
o Production-level compositing scripts – Example 1
o Production-level compositing scripts – Example 2
Chapter 6 | Entry-Level Roles and Job Applications at Top Visual Effects Studios
Index
Biography
Dr Jin Zhi is a tenured faculty member and a practitioner of film visual effects based in the United States. He is a Certified Nuke Trainer accredited by Foundry and a Teaching Fellow recognized by Advance HE in the UK. Over the past 15 years, he has worked as both a full-time academic and a VFX artist at leading universities and world-renowned visual effects studios in the United Kingdom and the United States.
