Nice props from The Futurum Group:
Here is why: Adobe Firefly is the most commercially successful generative AI product ever launched. Since it was introduced in March in beta and made generally available in June, at last count in October, Firefly users have generated more than 3 billion images. Adobe says Firefly has attracted a significant number of new Adobe users, making it hard to imagine that Firefly is not aiding Adobe’s bottom line.
Hey gang—here’s to having a great 2024 of making the world more beautiful & fun. Here’s a little 3D creation (with processing courtesy of Luma Labs) made from some New Year’s Eve drone footage I captured at Gaviota State Beach. (If it’s not loading for some reason, you can see a video version in this tweet).
Dig in, friends. 🙂
It’s a little nerdy even for my blood, but some of my teammates swear by these techniques that enable connecting Photoshop to a hosted instance of Stable Diffusion, enabling one to guide the process via a Photoshop doc and/or custom-trained styles:
Hah—I can dig this finger-rich pin from Pictoplasma.
What if your tiny planet—a visual genre I’ve enjoyed beating halfway into the ground—were our actual planet? Insta360, on whom I’ve spent crazy amounts of money buying brilliant-if-maddening gear, has now sent their devices to the edge of space:
Let’s talk vector generation, 3D mockup support, image-to-type, and more. Take ‘er away, Deke!
I mean, seriously, what even is all this?? I can’t explain; just please watch.
I’ve had way too much fun creating custom Lego sets based on friends’ & family’s rides, so to help others do it, I’ve made my first custom GPT, “Baby You Can Drive My Bricks.” Take it for a spin & let me know what you create!
It’s ludicrous to think that these folks formed the company just six months ago, and even more ludicrous to see what the model can already do—from video synthesis, to image animation, to inpainting/outpainting:
Our vision for Pika is to enable everyone to be the director of their own stories and to bring out the creator in each of us. Today, we reached a milestone that brings us closer to our vision. We are thrilled to unveil Pika 1.0, a major product upgrade that includes a new AI model capable of generating and editing videos in diverse styles such as 3D animation, anime, cartoon and cinematic, and a new web experience that makes it easier to use. You can join the waitlist for Pika 1.0 at https://pika.art.
This tech—or something much like it—is going to be a very BFD. Imagine simply describing the change you’d like to see in your image—and then seeing it.
[Generative models] still face limitations when it comes to offering precise control. That’s why we’re introducing Emu Edit, a novel approach that aims to streamline various image manipulation tasks and bring enhanced capabilities and precision to image editing.
Emu Edit is capable of free-form editing through instructions, encompassing tasks such as local and global editing, removing and adding a background, color and geometry transformations, detection and segmentation, and more. […]
Emu Edit precisely follows instructions, ensuring that pixels in the input image unrelated to the instructions remain untouched. For instance, when adding the text “Aloha!” to a baseball cap, the cap itself should remain unchanged.
And for some conceptually related (but technically distinct) ideas, see previous: Iterative creation with ChatGPT.
On the off chance you missed me over the last week or so, it’s due to my being off in Illinois with the fam, having fun making silliness like this:
Here’s a great look at how the scrappy team behind Luma.ai has helped enable beautiful volumetric captures of Phoenix Suns players soaring through the air:
Go behind the scenes of the innovative collaboration between Profectum Media and the Phoenix Suns to discover how we overcame technological and creative challenges to produce the first 3D bullet time neural radiance field NeRF effect in a major sports NBA arena video. This involved not just custom-building a 48 GoPro multi-cam volumetric rig but also integrating advanced AI tools from Luma AI to capture athletes in stunning, frozen-in-time 3D visual sequences. This venture is more than just a glimpse behind the scenes – it’s a peek into the evolving world of sports entertainment and the future of spatial capture.
If you keep hearing about “Gaussian Splatting” & wondering “WTAF,” check out this nice primer from my buddy Bilawal:
There’s also Two-Minute Papers, offering a characteristically charming & accessible overview:
It’s always great to learn from the master—especially when he’s making “spaghetti western” literal!
I’m really digging the experience of (optionally) taking a photo, feeding it into ChatGPT, and then riffing my way towards an interesting visual outcome. Here’s a gallery in which you can see some of the journeys I’ve undertaken recently.
Hah! This is my kind of ridiculous Adobe social content. 🙂 Happy Friday.
Man, I’m inspired—and TBH a little jealous—seeing 14yo creator Preston Mutanga creating amazing 3D animations, as he’s apparently been doing for fully half his life. I think you’ll enjoy the short talk he gave covering his passions:
The presentation will take the audience on a journey, a journey across the Spider-Verse where a self-taught, young, talented 14-year-old kid used Blender, to create high-quality LEGO animations of movie trailers. Through the use of social media, this young artist’s passion and skill caught the attention of Hollywood producers, leading to a life-changing invitation to animate in a new Hollywood movie.
Speaking of increasing resolution, check out this sneak peek from Adobe MAX:
It’s a video upscaling tool that uses diffusion-based technology and artificial intelligence to convert low-resolution videos to high-resolution videos for applications. Users can directly upscale low-resolution videos to high resolution. They can also zoom-in and crop videos and upscale them to full resolution with high-fidelity visual details and temporal consistency. This is great for those looking to bring new life into older videos or to prevent blurry videos when playing scaled versions on HD screens.
Interesting recent finds:
If you’ve been sleeping on Text to Vector, check out this handful of quick how-to vids that’ll get you up to speed:
Check out this quick demo of Illustrator’s new text-to-vector & mockup tools working together:
AI generated Logos onto any surface. pic.twitter.com/qY4tEkVK0Q
— Riley Brown (@rileybrown_ai) October 29, 2023
Directly sketch inside a 360º canvas, then generate results:
And see also the styles these folks are working to bring online:
Easy as ABC, 123?
Project Glyph Ease uses generative AI to create stylized and customized letters in vector format, which can later be used and edited. All a designer needs to do is create three reference letters in a chosen style from existing vector shapes or ones they hand draw on paper, and this technology automatically create the remaining letters in a consistent style. Once created, designers have flexibility to edit the new font since the letters will appear as live text that can be scaled, rotated or moved in the project.
Can you imagine something like this running in Photoshop, making it possible to re-pose objects and then merge them back into one’s scene?
The week before MAX, my teammate Christine had a bit of a cough, and folks were suddenly concerned about the Project Primrose sneak: it’d be awfully hard to swap out presenters when the demo surface is a bespoke dress made by & for exactly one person. Thankfully good health prevailed, and she was able to showcase Project Primrose:
Here’s a bit more info about the tech:
We propose reflective light-diffuser modules for non-emissive flexible display systems. Our system leverages reflective-backed polymer-dispersed liquid crystal (PDLC), an electroactive material commonly used in smart window applications. This low-power non-emissive material can be cut to any shape, and dynamically diffuses light. We present the design & fabrication of two exemplar artifacts, a canvas and a handbag, that use the reflective light-diffuser modules.
We’re looking to meet great PMs, engineers, data scientists, and more; come check out open roles!
Marc Levoy (professor emeritus at Stanford) was instrumental in delivering the revolutionary Night Sight mode on Pixel 3 phones—and by extension on all the phones that quickly copied their published techniques. After leaving Google for Adobe, he’s been leading a research team that’s just shown off the reflection-zapping Project See Through:
Today, it’s difficult or impossible to manually remove reflections. Project See Through simplifies the process of cleaning up reflections by using artificial intelligence. Reflections are automatically removed, and optionally saved as separate images for editing purposes. This gives users more control over when and how reflections appear in their photos.
Matthew Vandeputte used a mix of Generative Fill and Content-Aware Fill (or both) to make these rad little animations in After Effects:
[Via Tom Hightower]
Lens Blur, HDR, Point Color, and more: Katrin Eismann breaks down the update in this overview, and Matt Kloskowski shows the features in action here:
TBH I still kinda want one. 🙂
My old teammates Richard Tucker, Noah Snavely, and co. have been busy. Check out how their Generative Image Dynamics work makes it possible to interactively add small, periodic motion to photos:
I got to spend time Friday live streaming with the Firefly community, showing off some of the new MAX announcements & talking about some of what might be coming down the line. I hope you enjoy it, and I’d welcome any feedback on this session or on what you’d like to see in the future.
Short & sweet:
Adobe Illustrator has this feature called Retype (beta). With it you can select an image in Illustrator and enter Retype (beta) to determine the fonts that were used (at least close matches) in the JPG! It will also do the same for text that has been outlined. It’s amazing!
This tech lets you use augment text-based instructions with visual hints, such as rough sketches and paint strokes. Draw & Delight then uses Firefly to generate high-quality vector illustrations or animations in various color palettes, style variations, poses and backgrounds.
I got to spend 30 minutes chatting with educator & author Matt Miller last week, riffing on some tough but important questions around weighty, fascinating stuff like what makes us human, what we value around creativity, and how we can all navigate the creative disruptions that surround us.
Hear how Adobe generative AI solutions are designed to continually evolve, develop, and empower educators and students from kindergarten to university level. Generative AI is expected to have a significant impact on the creativity of students. It has the potential to act as a powerful tool that can inspire and enhance the creative process by generating new and unique ideas. Join Matt Miller, author and educator, and John Nack, principal product manager at Adobe, for this exciting discussion.
In this session, you’ll:
- Learn how Adobe approaches generative AI
- Hear experts discuss how AI affects teaching and learning
- Discover how AI can make learning more personalized and accessible
That’s the promise of Adobe’s Project Neo—which you can sign up to test & use now! Check out the awesome sneak peek they presented at MAX:
Incorporating 3D elements into 2D designs (infographics, posters, logos or even websites) can be difficult to master, and often requires designers to learn new workflows or technical skills.
Project Neo enables designers to create 2D content by using 3D shapes without having to learn traditional 3D creation tools and methods. This technology leverages the best of 3D principles so designers can create 2D shapes with one, two or three-point perspectives easily and quickly. Designers using this technology are also able to collaborate with their stakeholders and make edits to mockups at the vector level so they can quickly make changes to projects.
Wes Anderson & crew are back to making delightful miniature worlds, this time for “The Wonderful Story of Henry Sugar.” Enjoy three charming minutes, won’t you?
Back in the day, I dreaded demoing Photoshop ahead of the After Effects team: we’d do something cool, and they’d make that cool thing move. I hear echoes of that in Project Fast Fill—generative fill for video.
Project Fast Fill harnesses Generative Fill, powered by Adobe Firefly, to bring generative AI technology into video editing applications. This makes it easy for users to use simple text prompts to perform texture replacement in videos, even for complex surfaces and varying light conditions. Users can use this tool to edit an object on a single frame and that edit will automatically propagate into the rest of the video’s frames, saving video editors a significant amount of texture editing time.
Check it out:
Roughly 1,000 years ago (i.e. this past April!), I gave an early sneak peek at the 3D-to-image work we’ve been doing around Firefly. Now at MAX, my teammate Yi Zhou has demonstrated some additional ways we could put the core tech to work—by adding posable humans to the scene.
Project Poseable makes it easy for anyone to quickly design 3D prototypes and storyboards in minutes with generative AI.
Instead of having to spend time editing the details of a scene — the background, different angles and poses of individual characters, or the way the character interacts with surrounding objects in the scene — users can tap into AI-based character posing models and use image generation models to easily render 3D character scenes.
Check it out:
Here’s a fun little tutorial from my teammate Kris on using reference images to style your prompt (in this case, her pet turtle Pablo). And meanwhile, here’s a little gallery of good style reference images (courtesy of my fellow PM Lee) that you’re welcome to download and use in your creations.
Tutorial: How to generate images in different styles using reference image library in Adobe Firefly
It’s a new feature! Write a prompt and experiment with styles from the library or upload your own as a reference.
#AdobeMAX #CommunityxAdobeTry here:https://t.co/c9g7CGiuBU pic.twitter.com/cSt3cmMNR3
— Kris Kashtanova (@icreatelife) October 12, 2023
I’m really happy & proud that Firefly now enables uploading your own images & mixing them into your creations. For months & months, this has been users’ number 1 feature request.
But with power comes responsibility, of course, and we’ve spent a lot of time thinking about ways to discourage misuse of the tech (i.e. how do we keep this from becoming a rip-off engine?). I’m glad to say that we’ve invested in some good guidelines & guardrails:
First, we require users to confirm they have the right to use any work that they upload to Generative Match as a reference image.
Second, if an image’s Content Credentials include tags indicating that the image shouldn’t be used as a style reference, users won’t be able to use it with Generative Match. We will be rolling out the ability to add these tags to assets as part of the Content Credentials framework within our flagship products.
Third, when a reference image is used to generate an asset, we save a thumbnail of the image to help ensure that the use of Generative Match meets our terms of service. We also note that a reference image was used in the asset’s Content Credentials. Storing the reference image provides an important dose of accountability.
To be clear, these protections are just first steps, and we plan to do more to strengthen protections. In the meantime, your feedback is most welcome!
I’m delighted to say that the first Firefly Vector model is now available (as a beta—feedback welcome!) in Illustrator. Just download your copy to get started. Here’s a quick tour:
And more generally, it’s just one of numerous enhancements now landing in Illustrator. Check ’em out:
Hey everyone—I’m just back from Adobe MAX, and hopefully my blog is back from some WordPress database shenanigans that have kept me from posting.
I don’t know what the site will enable right now, so I’ll start by simply pointing to a great 30-second tour of my favorite new feature in Firefly, Generative Match. It enables you to upload your own image as a style reference, or to pick one that Adobe provides, and mix it together with your prompt and other parameters.
You can then optionally share the resulting recipe (via “Copy link” in the Share menu that appears over results), complete with the image ingredient; try this example. This goes well beyond what one can do with just copying/pasting a prompt, and as we introduce more multimodal inputs (3D object, sketching, etc.), it’ll become all the more powerful.
All images below were generated with the following prompt: a studio portrait of a fluffy llama, hyperrealistic, shot on a white cyclorama + various style images: