Technology Meets Filmmaking in Virtual Production with Paul Debevec Ep 27

Paul Debevec is Netflix’s Director of Research for Creative Algorithms and Technology, overseeing R&D for visual effects and virtual production with computer vision, graphics, and machine learning.

In 2002, his Light Stage 3 system surrounded actors with LEDs displaying images of virtual sets and environments for lighting-accurate compositing. Techniques from Paul’s work have been used to create key visual effects sequences in The Matrix, Spider-Man 2, Benjamin Button, Avatar, Gravity, Furious 7, Blade Runner: 2049, Gemini Man, Free Guy, and numerous video games.

Paul’s techniques for HDR imaging, image-based lighting, and photoreal digital actors have been recognized with two Academy Awards and the SMPTE Progress Medal. Paul is a Governor of the Visual Effects Branch of the Academy of Motion Picture Arts and Sciences, a Fellow of the Visual Effects Society, and an Adjunct Research Professor at the University of Southern California Computer Science.

More info at:

Highlights of the Episode:
0:00 Introduction
1:52 Paul’s early days
7:39 Image based lighting and HDR
12:44 Some early adaptations
17:19 3D rendering in natural light
19:01 Rendering Synthetic Objects into Real Scenes
23:04 The Campanelli movie
27:56 Facade: Reconstructing polyhedra models
32:16 High dynamic range imaging
34:52 Light Stage: 360° of illumination
39:47 Real world image based lighting
42:08 Virtual HRI Maps software named after Arnold
46:52 LED lighting environment from 20 years ago
52:49 Optimal linear color correction matrices
55:53 Working on Light fields with Google
59:27 Exciting future for filmmaking
01:02:18 Democratizing new technology
01:05:02 Advice for aspiring filmmakers
01:09:17 Go to a Siggraph conference near you


“When I look at the kind of work that made Star Wars possible, that made The Matrix possible, that made Gravity possible. It's folks who had this creative vision and realized maybe the technology can meet it there; but then they really took technology where there wasn't any real solid reason to believe it was gonna be possible to do it. And they just made it happen anyway.” - Paul Debevec

“The goal is to try to create tools for filmmakers and every so often this beautiful marriage of art and technology comes together. There's things ripe for the picking that maybe no one's quite seen yet that they're gonna get, and it's gonna blow everybody away and it's gonna be the next Matrix or the next Gravity.”
- Paul Debevec

“LED panels that have a wider color gamut have worse color rendition, they light stuff funnier because they have to put the RGB LEDs further out in weird parts of the spectrum to do that. But it dramatically increases the polarization. I think we're not too far away from people releasing panels that will have that additional color and it, and it's probably gonna be white and we know how to drive it.”- Paul Debevec

“You can invert that matrix and apply the inverse of that matrix to your in-camera pixels in the frustum so that your frustum now looks like it has bad color rendition. So without having to separate foreground and background, you put it all through that post correction matrix. It fixes considerably three quarters of color rendition and skin looks a heck of a lot better.” - Paul Debevec

Connecting with the Guest:

Connecting with CG Pro:

RSVP here for upcoming CG Pro Podcasts: