{"id":21936,"date":"2024-07-26T10:31:31","date_gmt":"2024-07-26T17:31:31","guid":{"rendered":"http:\/\/jnack.com\/blog\/?p=21936"},"modified":"2024-07-26T10:33:25","modified_gmt":"2024-07-26T17:33:25","slug":"neural-rendering-neo-firefly","status":"publish","type":"post","link":"http:\/\/jnack.com\/blog\/2024\/07\/26\/neural-rendering-neo-firefly\/","title":{"rendered":"Neural rendering: Neo + Firefly"},"content":{"rendered":"\n<p>Back when we launched Firefly (<em>alllll<\/em> the way back in March 2023), we hinted at the potential of combining 3D geometry with diffusion-based rendering, and I tweeted out a very early sneak peek:<\/p>\n\n\n<blockquote class=\"twitter-tweet\">\n<p dir=\"ltr\" lang=\"en\">Did you see this mind blowing Adobe ControlNet + 3D Composer Adobe is going to launch! It will really boost creatives\u2019 workflow. Video through <a href=\"https:\/\/twitter.com\/jnack?ref_src=twsrc%5Etfw\">@jnack<\/a><\/p>\n<p><a href=\"https:\/\/t.co\/8DL64XYgqU\">pic.twitter.com\/8DL64XYgqU<\/a><\/p>\n<p>\u2014 Kris Kashtanova (@icreatelife) <a href=\"https:\/\/twitter.com\/icreatelife\/status\/1657832224060846086?ref_src=twsrc%5Etfw\">May 14, 2023<\/a><\/p>\n<\/blockquote>\n<p><script async=\"\" src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<p>A year+ later, I&#8217;m no longer working to integrate the Babylon 3D engine into Adobe tools\u2014and instead I&#8217;m working directly with the Babylon team at Microsoft (!). Meanwhile I like seeing how my old teammates are continuing to explore integrations between 3D (in this case, project Neo). Here&#8217;s one quick flow:<\/p>\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"Stylize 3D Scenes with Firefly and Project Neo\" width=\"604\" height=\"340\" src=\"https:\/\/www.youtube.com\/embed\/0kqfrWzuw2U?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<p>Here&#8217;s a quick exploration from the always-interesting Martin Nebelong:<\/p>\n\n\n<blockquote class=\"twitter-tweet\">\n<p dir=\"ltr\" lang=\"en\">A very quick first test of Adobe Project Neo.. didn&#8217;t realize this was out in open beta by now. Very cool!<\/p>\n<p>I had to try to sculpt a burger and take that through Krea. You know, the usual thing!<\/p>\n<p>There&#8217;s some very nice UX in NEO and the list-based SDF editing is awesome.. very\u2026 <a href=\"https:\/\/t.co\/e3ldyPfEDw\">pic.twitter.com\/e3ldyPfEDw<\/a><\/p>\n<p>\u2014 Martin Nebelong (@MartinNebelong) <a href=\"https:\/\/twitter.com\/MartinNebelong\/status\/1783978307098136754?ref_src=twsrc%5Etfw\">April 26, 2024<\/a><\/p>\n<\/blockquote>\n<p><script async=\"\" src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n\n\n<p>And here&#8217;s a fun little Neo->Firefly->AI video interpolation test from Kris Kashtanova:<\/p>\n\n\n<blockquote class=\"twitter-tweet\">\n<p lang=\"en\" dir=\"ltr\">Tutorial: Direct your cartoons with Project Neo + Firefly + ToonCrafter <\/p>\n<p>1) Model your characters in Project Neo <br>2) Generate first and last frame with Firefly + Structure Reference <br>3) Use ToonCrafter to make a video interpolation between the first and the last frame <\/p>\n<p>Enjoy! <a href=\"https:\/\/t.co\/YPy32hoVDR\">pic.twitter.com\/YPy32hoVDR<\/a><\/p>\n<p>\u2014 Kris Kashtanova (@icreatelife) <a href=\"https:\/\/twitter.com\/icreatelife\/status\/1797710186989191399?ref_src=twsrc%5Etfw\">June 3, 2024<\/a><\/p><\/blockquote>\n<p> <script async=\"\" src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>","protected":false},"excerpt":{"rendered":"<p>Back when we launched Firefly (alllll the way back in March 2023), we hinted at the potential of combining 3D geometry with diffusion-based rendering, and I tweeted out a very early sneak peek: Did you see this mind blowing Adobe ControlNet + 3D Composer Adobe is going to launch! It will really boost creatives\u2019 workflow. [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"","sticky":false,"template":"","format":"standard","meta":[],"categories":[18,71,66],"tags":[],"_links":{"self":[{"href":"http:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/posts\/21936"}],"collection":[{"href":"http:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/comments?post=21936"}],"version-history":[{"count":4,"href":"http:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/posts\/21936\/revisions"}],"predecessor-version":[{"id":22096,"href":"http:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/posts\/21936\/revisions\/22096"}],"wp:attachment":[{"href":"http:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/media?parent=21936"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/categories?post=21936"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/jnack.com\/blog\/wp-json\/wp\/v2\/tags?post=21936"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}