1st Edition
The VES Handbook of Virtual Production
Definition of Virtual Production / Types of Virtual Production
What Types of Scenes Work Well for Virtual Production
Greenscreen for Virtual Production
Chapter 2. Advantages and Disadvantages of Shooting in Virtual Production
Advantages and Disadvantage of Using LED Volumes for Production
Chapter 3. How the Virtual Production Supervisor and Producer Relate to Other Departments
Virtual Production Supervisor Relationships
How the VFX Team on a Virtual Production Show Interfaces with Other Departments
Production Design for Virtual Production - What Has Changed, What Has Not?
Chapter 4. How To Shoot and Edit Animation Using Live-Action Virtual Production
Introduction: The Creative Story Process
Performance and Motion Capture for Animation
Virtual Camera and Director’s Layout
Creative Editorial
Chapter 5. Visualization
What is Storyvis
Previs/Mocap Pipeline and Flow chart
VR Scouting
VCAM - Virtual Camera System Used During Prep
Techvis and Motion Control Techvis
Stuntvis
VP Tools for On-Set Visualization
Simulcam
Postvis
Chapter 6. What Roles Are Needed for Virtual Production?
Overview of Virtual Production Staffing and Organization
Producing Virtual Production
Chapter 7. VAD (Virtual Art Department)
Definition of VAD / Workflow Integration
Artists for Asset Creation/Sets
Building Assets in Game Engines/Unreal
Managing Asset Quality and Performance Needs for Virtual Production
Virtual Scouting with Production Design - VR System
Virtual Pre-lighting with the DP
Multi-user Virtual Location Scouting – Pre-production
What Are the Differences Between a Multi-User Virtual Scout and Other Scouting Processes?
Techvis for Art Departments
Optimization and Delivery
Digital Content Creation Software Used in VAD
Content Library - Digital Backlot of Assets
Version Control Software
Chapter 8. Digital Asset Creation
Building 3D Assets in a VFX Facility
Capturing Surfaces with Photogrammetry to Create Assets
Procedural Digital Asset Creation
Move AI - Markerless Motion Capture
Using Miniatures for Virtual Production
Preparing 2D Plates for Virtual Production
Chapter 9. Volumetric Capture for Humans
Overview of Volumetric Capture
Volumetric Capture / Considerations for Production - Wardrobe, Hair, Makeup, Blocking
Volumetric Capture with Actors for Virtual Production
Chapter 10. How to Make Characters for Use in Virtual Production
Smart Simulated Characters (Digital Extras)
MetaHumans: Creating Digital Humans for Production
Animating MetaHumans with Motion Capture
Chapter 11. How to Capture Environments for LED Walls
Photogrammetry and LiDAR Sets for VP
Photogrammetry Optimization
Plate Photography for Playback on LED Walls
Chapter 12. Process Work on LED Stage
General Understanding of Process Work
Break it Down Into Techniques
Chapter 13. Projects Shot on an LED Stage
LED Wall ICVFX
LED Volume - 3D Tracking Full Volume versus 2D Playback on Set
Chapter 14. Challenges and Limitations of Shooting in a Volume
Limitations To Be Aware of When Shooting In a Volume
Moiré, Viewing Angle, Banding, Latency, Artifacts, and Frame Sync
Color Gamut and Color Shift
Sound Dampening
Post Cleanup of the LED Wall
Chapter 15. Setting Up an LED Volume Stage - Permanent versus Pop Up
Setting up a Full Volume for LED Walls
Stage Designs
Mobile LED Stages and Walls
LED Display Wall - LED Modules, Processors, and Rigging
Display Considerations
Chapter 16. LED Display Technology and Hardware
Characteristics of LED Displays
Color Consistency and Predictability
LED Display Validation Protocols
What Are the Aspects of Choosing a Panel
Types of Tools That Connect Cameras, Monitors and LED Displays to Computers
Key Characteristics of LED Display Processors
LED, OLED, and Playback Servers
Chapter 17. Volume Control - Brain Bar Labor
Description of Labor types for the Brain Bar
Rendering and Control Computers
Chapter 18. Software for Virtual Production
Virtual Production Software
Types of Content for On-Set Virtual Production
On-Set Applications
Media Servers for Filmmaking
Chapter 19. Cameras for Virtual Production
Characteristics of Cameras for Virtual Production
LED Processing for Camera - Frame Remapping for ICVFX
LED Processing for Camera - GhostFrame
Calibration and Lens Metadata
Chapter 20. Camera Tracking for Virtual Production
Overview of Inside-Out and Outside-In Tracking Systems
Outside-In Tracking Systems
Physical Tracker - Inertial / Optical / Hybrid / Encoder Based Solutions
Chapter 21. Introduction to Color Management for LED Walls
The Definition for Matching
Rendering a Virtual Scene to Produce Linear Light
Creating a Signal from Linear Light
LED Processor Conversions
Photography of LED Panels
Preview of Camera Look in Real-Time Engine
Limitations of LED Wall Panels
Chapter 22. Art Dept Integration in the Digital and Physical World
Getting the Environment Ready for the Cinematographer and Lighting in the Volume
Blending the Physical and Digital World
Workflow of Shooting on LED Wall
Real-Time Compositing Between Physical and Digital World
Chapter 23. Lighting Types for Virtual Production (LED Volume)
DPs on Lighting for a LED Volume
Lighting for the Outer Frustum
Using the LED Wall as a Direct Reflection Lighting Tool
External Lighting on Stage versus LED Wall Lighting
On-Set Image Based Lighting (OS IBL) / Lights with Special Integration for Virtual Production
Lighting Control Systems, Pixel Mapping and Protocols with Virtual Production
Chapter 24. Epic Training (Unreal Engine)
Training In-Camera Visual Effects (ICVFX) for Virtual Production
Chapter 25. Unity Software - Overview/Aspects of Virtual Production
Introduction
World Building in Unity
Camera Tracking in Unity
Simulcam with GS Shooting - Live Set Composting
Ziva Dynamics- Photographic (Realistic) People, Animals, and Creatures
Chapter 26. The Future of Virtual Production
Overview of the Future
Beyond Flat Screens for On-Set Virtual Production Stages
Alternative Display Technology: Projection Based Volume Using White Cyc
High Resolution Depth (HD3D)
MicroLED - High-End Display and Resolution
Full Gamut Color- RGBW
NVIDIA Omniverse and Artificial Intelligence
What is Next? - Beyond the Volume
Acknowledgments
Appendix A: Charts and Technical Diagrams
Appendix B: : The Virtual Production Glossary
Biography
Susan Zwerman, VES, is an experienced Visual Effects Producer with a passion for cutting-edge film production. She is highly respected for her expertise in visual effects and virtual production budgeting and scheduling. As chair of the DGA UPM/AD VFX Digital Technology Committee, Susan organizes virtual production seminars to introduce members to this exciting and evolving new technology. Susan received the Frank Capra Achievement Award in recognition of career achievement and service to the industry and the Directors Guild of America in 2013. She is a member of the Academy of Motion Picture Arts and Sciences, the Producers Guild of America, the Directors Guild of America, and a member and Fellow of the VES.
Jeffrey A. Okun, VES, is an award-winning Visual Effects Supervisor who is more than conversant with virtual production. He is a member and Fellow of the VES and a member of The Academy of Motion Pictures, Arts and Sciences, the American Society of Cinematographers (ASC), the Television Academy, and the Editor’s Guild. Okun created visual effects tracking and bidding software in 1992 that is still in wide use within the industry today, as well as the revolutionary visual effects techniques dubbed the "PeriWinkle Effect" and the "Pencil Effect" – a predictive budgeting tool. He is also a noted 60s, 70s, and 80s rock and roll photographer.
