The latest generative models are great for demonstrations, but are they really going to change the way movies and TV shows are made? Not in the short term, according to film and visual effects experts. But in the long term, the changes could be beyond our imagination.
On a panel at SIGGRAPH in Denver, Nikola Todorovic (Dynamics of wonders), Freddy Chavez Olmos (Boxel Studio) and Michael Black (Mesh capeMax Planck Institute) looked at the potential for generative AI and other systems to change — but not necessarily improve — the way media is created today. Their consensus was that while we can fairly question the usefulness of these tools in the immediate future, the pace of innovation is such that we should be prepared for radical change at any point thereafter.
One of the first topics addressed was the impractical nature of current video generators.
Todorovic pointed out the “misperception of AI that it’s a one-click solution, that’s going to give you a final shot with visual effects, and that’s really impossible. Maybe we’ll get there, but if you don’t have editing capabilities, that black box isn’t going to give you much. What we’re seeing now is that the user experience is still being discovered – these research companies are just starting to learn the terms of 3D and cinematography.”
Black noted that language fundamentally lacks the ability to describe some of the most important aspects of visual creation.
“I mean, things like yoga poses, ballet poses — there are some classical things that we have names for, that we can define, but most of the things that we do, we don’t have names for,” she said. “And there’s a good reason for that: It’s because humans actually have within themselves a generative model of behavior. But I don’t have a generative model of behavior.” images “In my head, if I want to explain to you what I see, I can’t project it with my eyes, and I’m not a good enough artist to draw it for you. So I have to use words, and we have a lot of words to describe the visual world. But if I want to describe a particular movement to you, I don’t have to describe it with words, I just do it for you, and then your motor system sees me and it kicks in to understand it. And so, I think it’s a biological reason, a neuroscientific reason, why we don’t have words for all of our movements.”
This may sound a bit philosophical, but the upshot is that textual cueing systems for images have fundamental limitations in terms of their control. Even the hundreds of technical and artistic terms used daily on set and in post-production are inadequate.
Chavez Olmos said that being from Mexico, he had few opportunities to participate in the film world, because all the money and expertise was concentrated in Los Angeles. But he said AI expertise — and the demand for it — is more widely distributed. “I had to leave Mexico because I didn’t have opportunities there; now I can see that there is that same opportunity for people who don’t need to go abroad to do it.”
Black is concerned, however, that sudden access to these processes could have unintended consequences in the short term.
“You can give someone a powerful car, but that doesn’t make them a Formula One driver, does it? It’s a bit like what we have now. People are talking about how everyone is going to make movies. They’re going to be crap, honestly,” he said. “The issue of democratization is exactly what (Chávez Olmos) said, and the power is that maybe some new voice will get a chance that they wouldn’t otherwise have. But the number of people who make really good movies will remain small, in my opinion.”
“The real revolution,” he continued, “the real power of what we’re seeing in AI is that we’re going to see a whole new genre of entertainment, and I don’t know exactly what that’s going to look like. I predict it’s going to be somewhere between video games, film, and real life. The film industry is passive storytelling – I sit back and watch, it’s like theatre or a podcast. I’m the passive recipient of entertainment. But in our everyday lives, we tell each other stories, we chat about what we did on the weekend, etc. And that’s a very active kind of interactive storytelling.”
Before that happens, however, Chavez Olmos said he expects a more traditional acceptance curve for AI-generated images and actors.
“I think the reaction will be the same as when we saw the first ‘Final Fantasy’ movie or ‘The Polar Express’: There won’t be anything yet, but people will start to accept these movies,” he said. “And instead of a completely computer-generated movie, it will be a completely AI movie, which I think we’ll see even later this year. I think people will get over that, like, ‘Okay, this is AI,’ people will accept it.”
“The important thing,” Black said separately, “and Pixar taught us this very clearly: It’s all about the story. It’s all about connecting with the characters. It’s all about the heart. And if the movie has heart, it doesn’t matter if the characters are AI, I think people will enjoy the movie,” he said. “That doesn’t mean they don’t want human actors. There’s a thrill in knowing that they’re real humans like us, but so much better than us, seeing a human at the top of their game, it inspires us all, and I don’t think that’s going to go away.”