Google AI researchers just published a fascinating paper on their VideoPoet project, a large language model that has so-called “zero-shot” capabilities. The paper is titled VideoPoet: A Large Language Model for Zero-Shot Video Generation, and has over 30 co-authors. Zero-shot learning refers to when the LLM is trained on some text classifications but then is able to figure out other classifications on its own.
Google has a website showing the impressive capabilities of VideoPoet. The program can create beautiful videos from text prompts, or videos from existing images. Plus, the program enables one to edit the videos, including stylization and inpainting (or changes to particular spot directed by the human).
According to the authors, “We present empirical results demonstrating the model’s state-of-the-art capabili- ties in zero-shot video generation, specifically highlighting VideoPoet’s ability to generate high-fidelity motions.”
Take a look:
With these advances, AI text-to-video generation seems poised to become a common method of content creation.