VideoPoet

VideoPoet is a generative video tool. Google Research's VideoPoet turns any autoregressive language model into a video generator by tokenizing video, images and audio into one vocabulary. It covers text to video, image to video, stylization, inpainting and video to audio, with prompt driven camera moves and clip extension for longer scenes.

VideoPoet is a research project from Google that treats video generation as a language modeling problem. Images, video and audio are converted into discrete tokens through the MAGVIT V2 video tokenizer and the SoundStream audio tokenizer, placing every modality into a shared vocabulary that a text based language model can read and predict. The model then learns to forecast the next video or audio token in a sequence, which is how it produces clips rather than retrieving them. The demonstration site is aimed at researchers, film and motion design professionals, and technically curious creators evaluating where generative video stands. Out of the box it handles text to video, text to image, image to video, video frame continuation, inpainting and outpainting, stylization, and video to audio, and those tasks can be composed for zero shot results such as text to audio. Default outputs run about two seconds, with longer pieces built by feeding one second of prior footage back in to predict the next second, a loop the team repeats to reach any duration while keeping object identity intact. In practice the model rewards careful prompt craft. Style tags like digital art, pencil art or ink wash can be appended to a base prompt, and camera language such as arc shot, dolly zoom or FPV drone shot steers movement. Prompts can be revised across successive extensions to shift a scene mid clip, and interactive editing lets a user generate several unguided candidates, keep the strongest, then refine it with text. To prove the point, Google used Bard to script a short raccoon story, generated clips for each beat and stitched them into a YouTube Short. Against commercial video generators, VideoPoet reads as a research showcase rather than a product. There is no visible subscription tier, editor timeline or export pipeline on the site, so it competes on the breadth of modalities and motion fidelity demonstrated in its samples, not on workflow convenience. Anyone weighing it should treat the published results as evidence of capability and expect access to be constrained by research release terms.