Published : 24 Mar 2022, 11:17 AM
Updated : 03 Aug 2022, 11:39 AM
Now Mark Zuckerberg and other technologists are dreaming of a future in which we spend more of our time in immersive digital worlds, called the “metaverse.” And Perlman has advice for metaverse creators: Focus on the face.
What Perlman means is that in movies and sophisticated video games, ingenious technology and creative professionals have combined to create increasingly lifelike virtual creatures. They do that by focusing on detailed facial expressions and body movements.
Think about Gollum from the “Lord of the Rings” movies, many Marvel movie superheroes or Brad Pitt as Benjamin Button, a man who ages in reverse. They aren’t real, but we can connect to them and their emotions.
Perlman, a longtime inventor who also helped to develop digital video technology at Apple, said that what he has seen so far from the metaverse feels flat compared with fully fleshed out Hollywood and video game characters. He said that technologists should be laser-focused on re-creating the human face and emotional expression. It’s extremely difficult technology, but one that Perlman said is necessary to tackle to give virtual interactions a sense of intimacy.
“The face is the most subtly expressive part of the body,” Perlman said. “And the face is a part of the body that none of the VR/XR headsets accurately capture,” he said, referring to virtual reality goggles and technologies like Snap’s Spectacles that mix digital images with the real world — often known by the umbrella term “extended reality” or XR.
That’s just one person’s opinion, and Perlman isn’t a disinterested observer. He is working on a wireless technology that he believes is a necessary first step for the metaverse or other immersive virtual interactions.
I wanted to talk to Perlman to mine lessons from the long marriage of technology and imagination in Hollywood and beyond. Even if the metaverse never pans out, we’re spending more time interacting through screens, and we’re hunting for more authentic ways to connect. It’s both a technological challenge and a human one.
Perlman has a history of helping to bring virtual characters to life. In the mid-2000s, his startup made a breakthrough that helped us feel connected to two-dimensional characters. The Mova Contour system used phosphorescent makeup on actors’ faces and custom-made camera gear and software to turn those performances into computer data.
This and similar approaches, now typically called motion capture, have become fairly common in movies and sophisticated video games to create fantasies that feel realistic. Those of us who have laughed at and cried along with characters who aren’t human owe a debt to Perlman and other technologists. (Perlman’s firm is involved in litigation that claims that other companies improperly profited from its intellectual property.)
If you’ve tried virtual worlds like Fortnite, Second Life or Meta’s Horizon Worlds, you know that the digital beings there don’t yet look or feel human. They’re cartoonish, and sometimes people in the metaverse don’t have legs. For now, that’s fine. But Perlman said that for immersive digital worlds to catch on, metaverse people will need to feel like people.
“In my view, the metaverse becomes interesting when it’s at least as good as advanced video games today, and likely even better,” he said. “It won’t have a wide appeal until it is a universe of humans.”
That’s where Perlman imagines playing a role. His company has worked for years on a wireless technology that promises to handle a cascade of digital information — including from virtual reality — and to translate within fractions of a second a head movement of someone wearing VR goggles into that same action on screen.
We’ll see if Perlman’s latest invention takes off, but there is a long history of creative ideas being enabled by bedrock and often dull technology inventions.
Without Amazon’s cloud computing service and others like it, there would have been no iPhone age. Without advances to shrink down computer chip components or put GPS in every smartphone, most of the technologies that we use daily wouldn’t be possible.
Perlman looks back at his work with Hollywood characters and hopes that a similar breakthrough is possible within this decade for virtual humans that we can feel in our bones.
© 2022 The New York Times Company