Meta’s new ‘Movie Gen’ AI model can create realistic video with audio clips
by Sunita · KalingaTVMeta has a new AI model for its user called Movie Gen. The new cutting-edge AI model has the ability to create highly realistic video and audio clips as per a user’s prompts. Meta aims to rival the Artificial Intelligence tools like OpenAI and ElevenLabs with this new AI model. The company claims it will make a significant development in AI-generated media.
According to Meta, Movie Gen can produce video clips up to 16 seconds long and audio clips up to 45 seconds long, generating both visuals and sound effects that sync seamlessly with the content. In a blog post, the company showcased examples of the tool’s capabilities, including videos of animals swimming and people performing creative tasks like painting on canvas. One demonstration even showed the AI inserting pom-poms into the hands of a man running through the desert, while another altered a dry parking lot into a puddle-filled scene with a skateboarder.
Movie Gen isn’t limited to creating new content; it can also be used to edit existing videos, enhancing creative possibilities for content creators. The AI tool also generates background music and sound effects, further elevating the production quality of the clips.
Meta highlighted data from blind tests that suggest Movie Gen performs favourably against offerings from competitors such as Runway, OpenAI, ElevenLabs, and Kling. However, unlike its Llama large-language models, Meta has no plans to release Movie Gen for open developer use, citing the need to evaluate risks individually for each model.
Instead, Meta plans to collaborate directly with the entertainment industry and content creators, incorporating the tool into its products sometime next year. The model was built using a mix of licensed and publicly available datasets, Meta said.
The announcement comes at a time when generative AI video technology is sparking both excitement and concern in Hollywood. Filmmakers are eager to explore how tools like Movie Gen can streamline production and bring innovative ideas to life. However, many are wary of AI systems trained on potentially copyrighted works, raising ethical and legal questions. Similar controversies have already surrounded OpenAI’s Sora model, particularly after accusations from actress Scarlett Johansson that her voice was imitated without permission.
While OpenAI has been in talks with Hollywood studios about potential partnerships, no deals have yet been finalised. Meanwhile, Lions Gate Entertainment, known for “The Hunger Games” and “Twilight,” has already partnered with AI startup Runway, granting access to its film library in exchange for the ability to use the trained model in future productions.