I’m told that the video is sped up by 4x, but still, it’s a pretty darn impressive application of ControlNet techniques to Stable Diffusion:
I’m reminded of Nathan Shipley’s cool GAN-powered drawing experiments from a couple of years ago:
Category Archives: ControlNet
My interview & demos with Deke
Hah, OMG:
- No one should be this excited to talk to me.
- Deke kindly & wildly overstates the scope of my role at Adobe.
But hey, what the hell, I’ll take it!
I had a lot of fun chatting with my old friend Deke McClelland, getting to show off a new possible module (stylizing vectors), demoing 3D-to-image, and more. Here, have at it if you must. 😅
00:00 Introducing John Nack
00:33 Adobe Firefly Preview #1: Discrete Asset Generation
02:39 One Day, Masked Assets?
04:01 A.I. Assets from the Libraries Panels?
05:32 ControlNet Is Wild
07:41 An A.I. Beer Commercial, from an Alien’s POV
08:19 patreon.com/dekenow Plug
08:39 Adobe Firefly Preview #2: 3D
10:51 Wrapping Your 3D Objects in A.I. Textures
11:56 Firefly Does Not Limit You to a Command-Line Prompt
13:06 Keeping It Safe for Work
15:17 Adobe Firefly Preview #3: Inpainting
16:52 Filling a Background with A.I.
17:38 Remove Is So Much Better Than Content-Aware Fill
19:41 When Can We Upload Custom SVG Logos to Type Effects?
20:40 Check Out Posts by Bilawal Sidhu
21:41 OTF Character Replacement with Wonder Studio
22:08 NeRF Turns Video into a 3D Object
23:29 Select Anything Meets Video
23:52 John’s Blog: jnack.com
24:16 Adobe Firefly Preview #4: Video
25:10 Could I Synthesize Sound FX? Story Boards?
25:38 The Unexpected Power of Cute 3D Characters
28:07 Wrapping Up with Deke and John
ControlNet + Blender = 🔥
I love this; just don’t make me download and learn Blender to use it. 😅
Use Stable Diffusion ControlNet in Photoshop
Check out this integration of sketch-to-image tech—and if you have ideas/requests on how you’d like to see capabilities like these get more deeply integrated into Adobe tools, lay ’em on me!
Also, it’s not in Photoshop, but as it made me think of the Photo Restoration Neural Filter in PS, check out this use of ControlNet to revive an old family photo:
3D + AI: Stable Diffusion comes to Blender
I’m really excited to see what kinds of images, not to mention videos & textured 3D assets, people will now be able to generate via emerging techniques (depth2img, ControlNet, etc.):
ControlNet is wild
This new capability in Stable Diffusion (think image-to-image, but far more powerful) produces some real magic. Check out what I got with some simple line art:
And check out this thread of awesome sauce:
Welcome to the meme-predicted future.
Photogrammetry -> ControlNet = Makeover Magic
My friend Bilawal Sidhu made a 3D scan of his parents’ home (y’know, as one does), and he recently used the new ControlNet functionality in Stable Diffusion to restyle it on the fly. Check out details in this post & in the vid below: