My old teammates Richard Tucker, Noah Snavely, and co. have been busy. Check out this quick video & interactive demo:
Excited to share our work on Generative Image Dynamics!
We learn a generative image-space prior for scene dynamics, which can turn a still photo into a seamless looping video or let you interact with objects in the picture. Check out the interactive demo:https://t.co/GLPBVpouJY pic.twitter.com/6h1Qq0kL2G
— Zhengqi Li (@zhengqi_li) September 15, 2023
According to the team, they trained the prior using a dataset of motion trajectories extracted from real-life video sequences that featured natural, oscillating motions like those seen in trees, flowers, candles, and wind-blown clothing. These trajectories can then be applied to convert static images into smooth-looping dynamic videos, slow-motion clips, or interactive experiences that allow users to interact with the elements within the image.