TikTok Owner’s New AI Tool Makes Lifelike Videos From A Single Photo – Forbes
OmniHuman sample videos demonstrate the scope of both human and animal movement. TikTok owner ByteDance has unveiled OmniHuman, an AI model that can generate lifelike videos of people talking, gesturing, posing, singing, playing instruments and more — from a single photo.“OmniHuman significantly outperforms existing methods, generating extremely realistic human videos based on weak signal inputs, especially audio,” reads the OmniHuman-1 project page. “It supports image inputs of any aspect ratio, whether they are portraits, half-body or full-body images, delivering more lifelike and high-quality results across various scenarios.” A research paper on the tool came out Sunday on online open-access archive arXiv.The model isn’t ready or available for public use, but researchers have shared sample videos showcasing its capabilities. Examples demonstrate hand and body movements seen from multiple angles, animated characters, animals and historical figures brought back to life.In one crisp black and white video, Albert Einstein speaks in front of a blackboard, punctuating his words with hand gestures and subtle facial expressions. It’s as if we’ve gone back in time to watch the famed theoretical physicist deliver a university lecture, only the footage looks like it was filmed today.“They’re very impressive,” Freddy Tran Nager, a clinical associate professor of communications at the University of Southern California’s Annenberg School for Communication and Journalism, said in an interview after viewing the sample videos. “If you were thinking of reviving Humphrey Bogart and casting him in a film, I’m not sure how that would look. But on a small screen, especially on a phone, these are impressive.”The tool places ByteDance, and therefore TikTok, squarely in the crowded race of competitors angling to create the best, most realistic-looking footage of AI-generated humans. These digital figures are popping up everywhere — as virtual influencers promoting products, government officials helping citizens navigate social services and faux versions of celebrities interacting with fans — or concerningly, appearing in fake political endorsements.Nager says he could envision a range of scenarios for a tool that generates videos from a single photo. For instance, students could choose how information in basic online courses gets delivered: “I would like Marilyn Monroe to teach me statistics. I want a muppet. I want Kermit the Frog.” Given OmniHuman’s connection to TikTok, Nager could also imagine it being used by burned-out content creators who turn to virtual versions of themselves for a respite.Or, he said, “TikTok can say, ‘You know what? Now we can just create videos on our own. Who needs the human beings?’”Samantha G. Wolfe — an adjunct professor at NYU’s Steinhardt School of Culture, Education and Human Development and founder of PitchFWD, an emerging-technology marketing consultancy — also sees both promise and potential peril in tools like OmniHuman.“Creating something from just a picture and making it look like it’s really talking and really moving is fascinating from a technological standpoint, but it could have a lot of potential negative consequences, too,” she said in an interview. “Pretend versions of business leaders or political leaders saying something that isn’t accurate can have a huge influence on a business, or a huge influence on a country.”As AI-generated videos grow ever more sophisticated, the risks also increase. “When it starts to look more and more like reality, more and more like humans actually doing it, the likelihood of people believing it becomes so much greater,” Wolfe said.The ByteDance team trained OmniHuman on more than 18,700 hours of human video data, combining multiple types of inputs, such as text, audio and physical poses.OmniHuman isn’t the first AI tool to generate videos from a single photo, but what sets it apart in Nager’s eyes is the amount of training data the team behind it has access to. “If you created a TikTok video,” he said, “there’s a good chance you’re now in a database that’s going to be used to create virtual humans.”A spokesperson for ByteDance clarified via email that the model didn’t train on data from the company’s products. The representative added that if OmniHuman is released for public use, it will include safeguards against harmful or misleading content, along with transparency measures to make it clear the content is AI-generated.One Community. Many Voices. Create a free account to share your thoughts. Our community is about connecting people through open and thoughtful conversations. We want our readers to share their views and exchange ideas and facts in a safe space.In order to do so, please follow the posting rules in our site’s Terms of Service. We’ve summarized some of those key rules below. Simply put, keep it civil.Your post will be rejected if we notice that it seems to contain:User accounts will be blocked if we notice or believe that users are engaged in:So, how can you be a power user?Thanks for reading our community guidelines. Please read the full list of posting rules found in our site’s Terms of Service.
