All posts by jnack

ChatGPT reimagines family photos

AI/ML, GPT-4o, Illustration

“Dress Your Family in Corduroy and Denim” — David Sedaris
“Turn your fam into Minecraft & GTA” — Bilawal Sidhu

And meanwhile, on the server side:

Google’s “Photoshop Killer”?

AI/ML

Nearly twenty years ago (!), I wrote here about how The Killing’s Gotta Stop—ironically, perhaps, about then-new Microsoft apps competing with Adobe. I rejected false, zero-sum framing then, and I reject it now.

Having said that, my buddy Bilawal’s provocative framing in this video gets at something important: if Adobe doesn’t get on its game, actually delivering the conversational editing capabilities we publicly previewed 2+ years ago, things are gonna get bad. I’m reminded of the axiom that “AI will not replace you, but someone using AI just might.” The same goes for venerable old Photoshop competing against AI-infused & AI-first tools.

In any case, if you’re interested in the current state of the art around conversational editing (due to be different within weeks, of course!), I think you’ll enjoy this deep dive into what is—and isn’t—possible via Gemini:

Specific topic sections, if you want to jump right to ’em:

  • 00:00 Conversational Editing with Google’s Multimodal AI
  • 00:53 Image Generation w/ LLM World Knowledge
  • 02:12 Easy Image Editing & Colorization 
  • 02:46 Advanced Conversational Edits (Chaining Prompts Together)
  • 03:37 Long Text Generation (Google Beats OpenAI To The Punch)
  • 04:25 Making Spicy Memes (Google AI Studio Safety Settings) 
  • 05:48 Advanced Prompting (One Shot ComfyUI Workflows) 
  • 07:19 Re-posing Characters (While Keeping Likeness Intact) 
  • 08:27 Spatial 3D Understanding (NO ControlNet) 
  • 10:42 Semantic Editing & In/Out Painting 
  • 13:46 Sprite Sheets & Animation Keyframes 
  • 14:40 Using Gemini To Build Image Editing Apps
  • 16:37 Making Videos w/ Conversational Editing

Happy birthday, Adobe Firefly

Adobe Firefly, AI/ML

The old (hah! but it seems that way) gal turns two today.

The ride has been… interesting, hasn’t it? I remain eager to see what all the smart folks at Adobe have been cooking up. As a user of Photoshop et al. for the last 30+ years, I selfishly hope it’s great!

In the meantime, I’ll admit that watching the video above—which I wrote & then made with the help of Davis Brown (son of Russell)—makes me kinda blue. Everything it depicts was based on real code we had working at the time. (I insisted that we not show anything that we didn’t think we could have shipping within three months’ time.) How much of that has ever gotten into users’ hands?

Yeah.

But as I say, I’m hoping and rooting for the best. My loyalty has never been to Adobe or to any other made-up entity, but rather to the spirit & practice of human creativity. Always will be, until they drag me off this rock. Rock the F on.

Adobe to offer access to non-Firefly models

Adobe Firefly, AI/ML

Man, I’m old enough to remember writing a doc called “Yes, And…” immediately upon the launch of DALL•E in 2022, arguing that of course Adobe should develop its own generative models and of course it should also offer customers a choice of great third-party models—because of course no single model would be the best for every user in every situation.

And I’m old enough to remember being derided for just not Getting It™ about how selling per-use access to Firefly was going to be a goldmine, so of course we wouldn’t offer users a choice. ¯\_(ツ)_/¯

Oh well. Here we are, exactly two years after the launch of Firefly, and Adobe is going to offer access to third-party models. So… yay!

Runway reskins rock

AI/ML, Illustration

Another day, another set of amazing reinterpretations of reality. Take it away Nathan…


…and Bilawal:

Mystic structure reference: Dracarys!

3D, AI/ML, Illustration

I love seeing the Magnific team’s continued rapid march in delivering identity-preserving reskinning

This example makes me wish my boys were, just for a moment, 10 years younger and still up for this kind of father/son play. 🙂

Behind the scenes: AI-augmented animation

AI/ML, Illustration

“Rather than removing them from the process, it actually allowed [the artists] to do a lot more—so a small team can dream a lot bigger.”

Paul Trillo’s been killing it for years (see innumerable previous posts), and now he’s given a peek into how his team has been pushing 2D & 3D forward with the help of custom-trained generative AI:”

Charmingly terrible AI-made infographics

AI/ML, Illustration, Infographics

A passing YouTube vid made me wonder about the relative strengths of World War II-era bombers, and ChatGPT quickly obliged by making me a great little summary, including a useful table. I figured, however, that it would totally fail at making me a useful infographic from the data—and that it did!

Just for the lulz, I then ran the prompt (“An infographic comparing the Avro Lancaster, Boeing B-17, and Consolidated B-24 Liberator bombers”) through a variety of apps (Ideogram, Flux, Midjourney, and even ol’ Firefly), creating a rogue’s gallery of gibberish & Franken-planes. Check ’em out.

Surrealism blooms through Pika

AI/ML

Check out this delightful demo:

Individual steps, as I understand them:

  • Generate image (in this example, using Google Imagen).
  • Apply background segmentation.
  • Synthesize a new background, and run what I think is a fine-tuned version of IC-Light (using Stable Diffusion) to relight the entire image, harmonizing foreground/background. Note that identity preservation (face shape, hair color, dress pattern, etc.) is very good but not perfect; see changes in the woman’s hair color, expression, and dress pattern.
  • Put the original & modified images into Pika, then describe the desired transformation (smooth transition, flowers growing, clouds moving, etc.).

NeRFtastic BAFTAs

3D, AI/ML

The British Academy Film Awards have jumped into a whole new dimension to commemorate the winners of this year’s awards:

The capturing work was led by Harry Nelder and Amity Studio. Nelder used his 16-camera rig to capture the recent winners. The reconstruction software was a combination of a cloud-based platform created by Nelder, which is expected to be released later this year, along with Postshot. Nelder further utilized the Radiance Field method known as Gaussian Splatting for the reconstruction. A compilation video of all the captures, recently posted by BAFTA, was edited by Amity Studio

[Via Dan Goldman]

Lego together creative AI blocks in Flora

AI/ML

Looks promising:

Their pitch:

  • Create workflows, not just outputs. Connect Blocks to shape, refine, and scale your creative process.
  • Collaborate in real time. Work like you would in Figma, but for AI-powered media creation.
  • Discover & clone workflows. Learn from top creatives, build on proven systems and share generative workflows inside FLORA’s Community.

Sigma BF: Clean AF

Design, Photography

Refreshingly simple design!

Is it for me? Dunno: lately the only thing that justifies shooting with something other than my phone is a big, fast zoom lens, and I don’t know whether pairing such a thing with this slim beauty would kinda defeat the purpose. Still, I must know more…

Here’s a nice early look at the cam plus a couple of newly announced lenses:

Perhaps image-to-3D was a mistake…

3D, AI/ML

Behold the majesty (? :-)) of CapCut’s new “Microwave” filter (whose name makes more sense if you listen with sound on):

https://youtube.com/shorts/bshQXczbZdw?si=aFwvtgs-fKf2wl8x

As I asked Bilawal, who posted the compilation, “What is this, and how can I know less about it?”

EditIQ edits single long shots into multiples virtual shots

AI/ML

Check it out (probably easier to grok by watching vs. reading a description):

From the static camera feed, EditIQ initially generates multiple virtual feeds, emulating a team of cameramen. These virtual camera shots termed rushes are subsequently assembled using an automated editing algorithm, whose objective is to present the viewer with the most vivid scene content.

Controlling video generation with simple props

AI/ML

Tired: Random “slot machine”-style video generation
Inspired: Placing & moving simple guidance objects to control results:
Check out VideoNoiseWarp:

Analog meets AI in the papercraft world of Karen X Cheng

Adobe Firefly, AI/ML, Generative Fill

Check out this fun mixed-media romp, commissioned by Adobe:

And here’s a look behind the scenes:

A cool Firefly image->video flow

Adobe Firefly, AI/ML

For the longest time, Firefly users’ #1 request was to use images to guide composition of new images. Now that Firefly Video has arrived, you can use a reference image to guide the creation of video. Here’s a slick little demo from Paul Trani:

Titles: Severance Season 2

Design

Building on the strong work from the previous season,

Berlin’s Extraweg have created… a full-blown motion design masterpiece that takes you on a wild ride through Mark’s fractured psyche. Think trippy CGI, hypnotic 3D animations, and a surreal vibe that’ll leave you questioning reality. It’s like Inception met a kaleidoscope, and they decided to throw a rave in your brain. [more]

Google Photos will flag AI-manipulated images

AI/ML, Google Photos

These changes, reported by Forbes, sound like reasonable steps in the right direction:

Starting now, Google will be adding invisible watermarks to images that have been edited on a Pixel using Magic Editor’s Reimagine feature that lets users change any element in an image by issuing text prompts.

The new information will show up in the AI Info section that appears when swiping up on an image in Google Photos.

The feature should make it easier for users to distinguish real photos from AI-powered manipulations, which will be especially useful as Reimagined photos continue to become more realistic.

DeepSeek meets Flux in Krea Chat

AI/ML

Conversational creation & iteration is such a promising pattern, as shown through people making ChatGPT take images to greater & greater extremes:


But how do we go from ironic laughs to actual usefulness? Krea is taking a swing by integrating (I think) the Flux imaging model with the DeepSeek LLM:

It doesn’t yet offer the kind of localized refinements people want (e.g. “show me a dog on the beach,” then “put a hat on the dog” and don’t change anything outside the hat area). Even so, it’s great to be able to create an image, add a photo reference to refine it, and then create a video. Here’s my cute, if not exactly accurate, first attempt. 🙂

A mind-blowing Gemini + Illustrator demo

AI/ML

Wow—check out this genuinely amazing demo from my old friend (and former Illustrator PM) Mordy:

In this video, I show how you can use Gemini in the free Google AI Studio as your own personal tutor to help you get your work done. After you watch me using it to learn how to take a sketch I made on paper to recreating a logo in Illustrator, I promise you’ll be running to do the same.

MatAnyone promises incredible video segmentation

AI/ML

What the what?

Per the paper,

We propose MatAnyone, a robust framework tailored for target-assigned video matting. Specifically, building on a memory-based paradigm, we introduce a consistent memory propagation module via region-adaptive memory fusion, which adaptively integrates memory from the previous frame. This ensures semantic stability in core regions while preserving fine-grained details along object boundaries. 

Premiere Pro now lets you find video clips by describing them

AI/ML

I love it: nothing too fancy, nothing controversial, just a solid productivity boost:

Users can enter search terms like “a person skating with a lens flare” to find corresponding clips within their media library. Adobe says the media intelligence AI can automatically recognize “objects, locations, camera angles, and more,” alongside spoken words — providing there’s a transcript attached to the video. The feature doesn’t detect audio or identify specific people, but it can scrub through any metadata attached to video files, which allows it to fetch clips based on shoot dates, locations, and camera types. The media analysis runs on-device, so doesn’t require an internet connection, and Adobe reiterates that users’ video content isn’t used to train any AI models.

Celebrating the skate art of Jim Phillips

Illustration

If you’re like me, you may well have spent hours of your youth lovingly recreating the iconic designs of pioneering Santa Cruz artist Jim Phillips. My first deck was a Roskopp 6, and I covered countless notebook covers, a leg cast, my bedroom door, and other surfaces with my humble recreations of his work.

That work is showcased in the documentary “Art And Life,” screening on Thursday in Santa Cruz. I hope to be there, and maybe to see you there as well. (To this day I can’t quite get over the fact that “Santa Cruz” is a real place, and that I can actually visit it. Growing up it was like “Timbuktu” or “Shangri-La.” Funny ol’ world.)

Gemini turns photos into interactive simulations (!)

AI/ML

Check out this wild proof of concept from Trudy Painter at Google, and click into the thread for details.

Quick fun with Krea, Flux, custom training, and 3D

3D, AI/ML

Putting the proverbial chocolate in the peanut butter, those fast-moving kids at Krea have combined custom model training with 3D-guided image generation. Generation is amazingly fast, and the results are some combo of delightful & grotesque (aka “…The JNack Story”). Check it out:

“The Heist,” conjured entirely in Google Veo

AI/ML

Here’s another interesting snapshot of progress in our collective speedrun towards generative storytelling. It’s easy to pick on the shortcomings, but can you imagine what you’d say upon seeing this in, say, the olden times of 2023?

The creator writes,

Introducing The Heist – Directed by Jason Zada. Every shot of this film was done via text-to video with Google Veo 2. It took thousands of generations to get the final film, but I am absolutely blown away by the quality, the consistency, and adherence to the original prompt. When I described “gritty NYC in the 80s” it delivered in spades – CONSISTENTLY. While this is still not perfect, it is, hands down, the best video generation model out there, by a long shot. Additionally, it’s important to add that no VFX, no clean up, no color correction has been added. Everything is straight out of Veo 2. Google DeepMind

SynthLight promises state-of-the-art relighting

AI/ML, Photography, Relighting

Here’s a nice write-up covering this paper. It’ll be interesting to dig into the details of how it compares to previous work (see category). [Update: The work comes in part from Adobe Research—I knew those names looked familiar :-)—so here’s hoping we see it in Photoshop & other tools soon.]

Krea introduces realtime 3D-guided image generation

3D, AI/ML

Part 9,201 of me never getting over the fact we were working on stuff like this 2 years ago at Adobe (modulo the realtime aspect, which is rad) & couldn’t manage to ship it. It’ll be interesting to see whether the Krea guys (and/or others) pair this kind of interactive-quality rendering with a really high-quality pass, as NVIDIA demonstrated last week using Flux.