Meta Enters the AI Video Chat with Movie Gen, A Comprehensive Model for Film-Makers

In the rapidly evolving AI video landscape, Meta has released a new model, Movie Gen, capable of generating movies from text prompts. This comprehensive model shares many features with OpenAI’s Sora model, including limitations on access, being restricted to specific filmmakers partnering with Meta rather than a public rollout.

Movie Gen is a powerful tool, demonstrated in the ability to produce movies from text prompts. It can create 16-second videos, upscale them to 1080p resolution, and even add audio soundtracks. However, its speed is limited to 16 frames per second (fps), which is slower than the standard 24 fps. This limitation can be overcome by reducing the video length to 10 seconds.

Movie Gen: The Action

Despite this limitation, Movie Gen is still an impressive tool, thanks to its ability to generate videos from text prompts and its flexibility. With the right prompt, users can create a variety of videos, from 10 seconds to 16 seconds long. The model’s text-based editing feature allows for precise adjustments to the video, such as changing clothing, backgrounds, and camera moves.

• Precision editing: Adjust camera angles, movements, and tracking

• New outfits, backgrounds, and locations: Change characters and settings

• Panning and tracking requests: Incorporate into video edits

The Audio AI Future

While impressive visuals are now common among AI video makers, Meta is focusing on the audio side of filmmaking. Movie Gen uses text prompts to generate soundtracks, including rain sounds, car engines, and music, which can be matched to the mood of the video. However, human speech is not currently part of Movie Gen’s capabilities.

Meta’s decision to keep Movie Gen away from the public might be due to concerns over misuse, as was the case with their AI song generator. Nevertheless, this move may have drawbacks, as other AI video generators, such as those from Runway, Pika, Stability AI, Hotshot, and Luma Labs’ Dream Machine, are already available or upcoming.