Savoring a tub of buttery popcorn while asking yourself how did they do that? can be half the fun of watching big cinematic films like Jurassic Park or Indiana Jones. Many of those burning behind-the-scenes questions and more were answered during the first season of the Lucasfilm docuseries Light & Magic on Disney+. Now returning for a second season, tech and movie buffs alike have the chance to get to know the very real visual effects artists who create magic at the movies and how they handled challenges during the dawn of digital filmmaking.
One of those technical wizards is John Knoll, who began his career with George Lucas’ Industrial Light & Magic in 1986. You might not know his name, but if you have ever used Photoshop, you can thank John and his brother Thomas for it. The duo co-wrote the original software during the 1980s, which would subsequently be used on the groundbreaking 1989 James Cameron film The Abyss. In the years since, Knoll has won an Academy Award for his team’s work bringing a ship of sea creature-pirate hybrids to life in 2006’s Pirates of the Caribbean: Dead Man’s Chest, in addition to five other nominations.
Popular Science recently spoke with Knoll about his work and career. Answers have been condensed for clarity.
Laura Baisas: How did your upbringing among scientists influence your career?
John Knoll: Growing up in a family of scientists, engineers, and medical professionals, I got a good strong dose of the scientific method and logical analysis. Things like how to analyze problems, break them down, and solve them. My dad modeled a lot of behavior that really influenced me a lot. Seeing him take up an interest in something, practicing, and getting to be good at it, and then applying that to a whole bunch of different fields. I kind of copied the same thing.
It was also a household that valued art as well. Even with this strong tech emphasis, the arts were valuable. I was probably heading for some kind of engineering field, but when Star Wars came out, I’m suddenly like, “Oh man, there’s, there’s some really exciting new stuff happening in filmmaking.” But if there is a field in filmmaking that is kind of engineering for film, that’s really what visual effects is. The mixture of almost equal parts of art and technology is a fusion that I find really very enjoyable. It allows me to exercise the engineer and scientist part of my brain, along with my love of art and expression.
LB: What is one part of working in visual effects people might find surprising?
JK: The scientific method is super useful and it applies to almost anything. Obviously, it applies to tech and engineering fields, but it also applies to art in ways that I don’t think people really appreciate. I am exercising that all the time by looking at some kind of problem that we’re facing and determining whether it’s an art problem or an aesthetic issue. Then, it is narrowing down where the problem is. Is this a motion problem, or is this a look problem? And then do I still see the problem if I stop it? You apply some of the same things you do to software debugging too. So, I’m applying all those same engineering principles to the art that I do, and it serves me really well.
[ Related: How ‘Rogue One’ resurrected the Death Star. ]
LB: Could you pinpoint a particularly memorable technical or artistic challenge and how you worked to overcome it?
JK: Onset motion capture is a big one. During the Star Wars prequels, the general methodology we were using for CG creatures was that we would cast an actor to play that role on set. We would do rehearsals, shoot a reference take with the actor in the frame. Then, we would have them step out and have the camera operator remember to frame it as they did the actors that were still in the shot and we’d put the CG character in. I appreciate that’s a really hard thing to ask, to act to nothing, and remember what you were doing. And they did their best.
On the first Pirates of the Caribbean movie, I started with that methodology. But I had a whole bunch of shots where we started with an actor in their picture costume [for the scene where the cursed crew of the Black Pearl turn into skeletons in moonlight] and we had to transition them into these CG skeletons. The only methodology that really made sense for those sequences was to just shoot the actors in their picture costumes, and when they went into the moonlight, we would match their motion and render them. We could then paint out any part that the CG skeleton didn’t cover up.
That methodology turned out to be better for everyone. It was much easier for the other actors in the scene, because they weren’t trying to remember where they were looking, and the eye lines were all right. And when the camera operators were framing up shots, they had all the elements they were composing with were present in frame, so it was better operation.
LB: What are your thoughts on AI’s role in visual effects?
JK: I think AI is going to be like an artist’s tools. I think the visions that people have of somebody just typing in a prompt and getting a finished shot are pretty unrealistic. The best way to overcome fear of that stuff is to actually use these tools and see how limited they are, because I think that right now, the tools that take simple prompts are gimmicks.
With what we have available to us right now, a skilled artist with a good eye and talent can make something amazing with those tools. And somebody who does not have that skill and talent makes terrible looking stuff for those tools. I already see that with the AI diffusion image generators. I see people who don’t have good taste make terrible looking stuff with them, and I see people with better taste making better looking stuff. So I think that’s only going to become more extreme when the level of control that is needed appears. People with good taste are going to make good looking compelling work. I don’t fear a future where skill and talent are not valued.
LB: Very serious question to wrap things up. Who is your favorite fictional scientist?
JK: I grew up with the original Star Trek TV show and I love Commander Scotty. I love when the clock is ticking and everything depends on him, he’s got to get the warp engines back online, and he’s in there splicing wires and all that stuff. He’s the hero of the show many times. I think that scientists and engineers are super important to the world and they should get the respect they deserve.
Lucasfilm’s Light & Magic season 2 debuts on Disney+ on April 17th.
More deals, reviews, and buying guides
