{"id":21518,"date":"2024-01-24T09:51:00","date_gmt":"2024-01-24T17:51:00","guid":{"rendered":"http:\/\/jnack.com\/blog\/?p=21518"},"modified":"2024-01-24T11:44:46","modified_gmt":"2024-01-24T19:44:46","slug":"google-introduces-lumiere-for-video-generation-editing","status":"publish","type":"post","link":"http:\/\/jnack.com\/blog\/2024\/01\/24\/google-introduces-lumiere-for-video-generation-editing\/","title":{"rendered":"Google introduces Lumiere for video generation &#038; editing"},"content":{"rendered":"\n<p>Man, not a day goes by without the arrival of some <a href=\"https:\/\/lumiere-video.github.io\/\">new &amp; mind-blowing magic<\/a>\u2014not a day!<\/p>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"Lumiere\" width=\"604\" height=\"340\" src=\"https:\/\/www.youtube.com\/embed\/wxLr02Dz2Sc?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<blockquote class=\"wp-block-quote\">\n<p>We introduce Lumiere &#8212; a text-to-video diffusion model designed for synthesizing videos that portray realistic, diverse and coherent motion &#8212; a pivotal challenge in video synthesis. To this end, we introduce a Space-Time U-Net architecture that generates the entire temporal duration of the video at once, through a single pass in the model. This is in contrast to existing video models which synthesize distant keyframes followed by temporal super-resolution &#8212; an approach that inherently makes global temporal consistency difficult to achieve. [&#8230;]<\/p>\n\n\n\n<p>We demonstrate state-of-the-art text-to-video generation results, and show that our design easily facilitates a wide range of content creation tasks and video editing applications, including image-to-video, video inpainting, and stylized generation.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"<p>Man, not a day goes by without the arrival of some new &amp; mind-blowing magic\u2014not a day! We introduce Lumiere &#8212; a text-to-video diffusion model designed for synthesizing videos that portray realistic, diverse and coherent motion &#8212; a pivotal challenge in video synthesis. To this end, we introduce a Space-Time U-Net architecture that generates the [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[66],"tags":[],"_links":{"self":[{"href":"http:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/posts\/21518"}],"collection":[{"href":"http:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/comments?post=21518"}],"version-history":[{"count":3,"href":"http:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/posts\/21518\/revisions"}],"predecessor-version":[{"id":21525,"href":"http:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/posts\/21518\/revisions\/21525"}],"wp:attachment":[{"href":"http:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/media?parent=21518"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/categories?post=21518"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/tags?post=21518"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}