Ten Cool Things I’ve Sent My Adobe Friends

AI/ML

“My love language is sending unwanted bespoke newsletters.” So said someone once, and damn if it’s not the same with me. I’m forever DM’ing interesting Twitter finds to friends working across the industry.

Just in case they’d be of interest to you, here are ten recent things I’ve forwarded to a couple of pals who are exploring AI & creativity at Adobe. I figure, the more we can compare notes, the better we can work together on behalf of creative people.

So, here goes—in simple reverse chronological order:

Google unveils Genie 4

3D, AI/ML

I can’t wait to see how tech like this totally upends image creation & editing as we know them. That’s just my hunch, mind you, but converting images into this kind of interactive world makes it seem almost inevitable.

From the official blog:

Today, we are announcing Genie 4, our new world model, which unlocks unprecedented realism and improves real-time interactive simulation. Genie 4 is natively multimodal, enabling audio-like music, ambient noises, and interactive dialogue to be simultaneously generated with the world.

The Genie 4 model incorporates and builds on capabilities from the Gemini family: Gemini Omni anchors the simulations in real-world appearance and behavior, while other Gemini models provide the complex reasoning needed to help maintain a simulation’s coherence and continuity.

With this foundation, you can set an event in motion, and Genie 4 simulates what happens next: how the world responds, how it sounds, and how the people in it move, react, and speak.

Understanding and simulating the world is a crucial aspect of building AGI. Genie 4 advances our ability to create safe, infinitely variable simulations that can help with everything from developing training environments for robots and agents to unlocking the creation of new interactive experiences.

Gemini Goes Goth

AI/ML

I got your cathedral right here…

Antigravity A1 360º drone looks rad

Photography

Back at the start of the pandemic, I spurged on what proved to be a fiddly, borderline unusable Insta360 camera that could break into chunks that were meant to fit around my DJI Mavic & give it 360º vision. The thing was a real turd, but I still love the wild virtual camera moves it could enable in post.

Now the company has stepped into the drone space with some sweet goggles (Ladies…) and a joystick used to control its ultralight new bird. I fully don’t need such a thing (sadly it’s rare to find myself in places that are both drone-worthy & drone-friendly), but man I think I’d enjoy using one.

Ideogram ships native vector generation

AI/ML, Illustration

I’m a longtime fan of Mohammad & team, and part of me still wishes I’d been able to convince Adobe to acquire them back in 2022. That didn’t happen, obviously, and we went on to fail to enable text-to-vector creation with Firefly in 2023 (and, of course, beyond). Now Ideogram has shipped their solution:

Here is what it made for an uploaded pic of my son.

AI lets your fingers do the animating

Miscellaneous

I got a kick (heh, oh no) out of this fun marriage between organic & imagined worlds. If only I’d had this tech in 8th grade, I’d have used it on my buddies’ compulsive finger-scale simulation of kick flips & other skateboard tricks.

The animator’s comments are pretty thoughtful, so in case Twitter won’t display its translation below, I’ve pasted it below the post:

Ever since I started dabbling in animation—and it’ll be four decades soon—I’ve been waiting for tools that let you animate in a more organic and contextual way.

The entire traditional technical overlay on computer animation is absurdly overcomplicated.

It worked a bit like a gatekeeper. “Look how terrifyingly difficult this is, better leave it to the specialists, because no one else will grasp the depth and complexity of this magical process.” In fact, there are few other industries that have turned the complication of tools into a virtue, while simultaneously making the process inhuman and unintuitive.

From the start of this fun, I’ve dreamed of something else. Building the dynamics of a scene or a character’s movement is hard on abstract Bézier curves, on multiply nested hierarchical skeletons, but it’s easy to show with a gesture, fingers, body, or even just a whistle.

Finally, it’s possible. People are multimodal, so it’s great that machines finally are too. Finally, instead of “move the mouth corner controller 5 mm up, and increase the eye squint, move the eyebrow controller up 3 mm”—you’ll be able to say “smile.” Which is what I wish for myself and for all of us.

Reconstructing ancient ruins in 3D with Gemini 4 Argon

3D, AI/ML

My wife & son are huge fans of LIDAR-heavy shows like Expedition Unknown, and I can only imagine how they and many others will use these new techniques to shed light on archeological finds:

“We don’t get to be heroes anymore…”

AI/ML, Idle Philosophizing

Man, I know I’ve been on a bit of a bummy kick with some AI-related posts lately, but I think we’ve gotta look at this maelstrom from all angles—including the really emotional ones.

I found this episode really touching:

Steven Strogatz, a professor of mathematics at Cornell University, describes the excitement and grief of watching A.I. take on work that gives people a sense of purpose — and asks what could be lost if machines deliver answers without the human struggle, insight and understanding that make solving problems meaningful.

Likening mathematical exploration to sex & courtship was not, I’ll admit, something I expected to hear—and yet it speaks to pursuit, triumph, and resultant meaning. Give it a listen, and I’ll bet you find a lot that resonates.

AI: “Grief, burnout, and opportunity”

AI/ML, Shit That Actually Matters

“I don’t want to steer, I want to effing row!“

In a world polarized between AI hype & hate, I found Molly Graham’s take refreshing. She of “Give away your Legos” fame (i.e. give away what you’ve gotten good at) plainly acknowledges how much it sucks when AI one-shots so much of your hard-won craft and skills.

Yes, big step-changes in our tools promise that we can go from building houses to building entire worlds, but… what’s wrong with building just houses? Why is my work, my accomplishment, no longer enough?

I think you’ll find some thoughtful insights in this candid chat she had with Lenny Rachitsky:

Gemini provided a nice summary of key related points in her talk:

Key Emotional Challenges

  • The Identity Shift: Many professionals are mourning the loss of their traditional craft, moving from “rowing” (hands-on execution) to “steering” (managing AI tools). As Molly notes, “I’ve heard from a lot of engineers just like I really miss what it used to be” (1:03-1:06).
  • Loneliness of Collaboration: The collaborative nature of work is being altered as humans increasingly interact with AI agents rather than teammates. Molly describes a conversation with a product leader who felt “it’s kind of lonely” because “I miss these parts of the collaboration that have been a little bit stripped away” (21:43-21:52).
  • Burnout and Pressure: With burnout rates rising (55% according to the podcast data), employees feel pushed to maintain high productivity while simultaneously managing the existential anxiety of a changing job landscape (29:20-30:05).

Critical Perspective

  • Normalization of Grief: Molly stresses, “Change sucks. It also can be awesome but we don’t have to be fluffy bunnies about this. We can also just say this is hard” (22:03-22:08).
  • The Importance of Leadership: She cautions that leaders who attempt to gloss over these human realities risk alienating their teams, urging managers to create space for open dialogue about the difficulty of this era (15:15-15:20).

LTX AlphaGen promises amazing background removal

AI/ML

“Dad, what were the 90’s like?”
“Well, son, we spent most of it slowly dying as we set vector points frame by frame in order to (badly) separate items from video backgrounds…”

Maybe now models like this can once & truly drive a stake through the hearts of such dark days. Check it out, along with some sharp words from Aharon Rabinowitz—who 1000% knows what he’s talking about:

The Pope on AI, art, and humanity

AI/ML

I’m not sure what this will translate into, concretely, but I’m glad to see his holiness raise the topic:

Runway’s reframing tech melts my brain

AI/ML, Illustration

OMG, it’s like Photoshop’s now-ancient (!) Content-Aware Scale, times 10. Check out the thread below, or look in the album I made to gather its six eye-popping examples.

“Love, Rendered”: Can AI remember it for you wholesale?

AI/ML

Honestly I have some very mixed feelings here: as the son of an elderly dad who’s facing increasing challenges around memory, I have a hard time even watching this trailer, much less the full film. Visualizing old memories, reanimating old friends… Is this a good idea? I have no idea, and emotionally I find I can’t much engage beyond the basic premise. Still, because it’s culturally interesting, and we’ll almost certainly see more explorations like this:

Ethelle and Burt have shared a lifetime together, but as Burt’s memory begins to fade, so too does the story of how their love began. Holding onto one cherished moment that was never photographed or recorded, Ethelle works with clinical experts and her family to recreate their first meeting using generative technology. LOVE, RENDERED is an intimate documentary short about love, memory, aging and the ways new tools may help preserve human connection. Directed by two-time Oscar-nominated filmmaker Liz Garbus and produced by Oscar winner Dan Cogan and Oscar-nominee Darren Aronofsky.

Interactive letter-plumping

Typography

Fun!

Seems there’s a whole Insta feed full of related delights, like this:

Netflix asks, “You wanna get high (dynamic range)?”

AI/ML, Image Science, Photography

Check out DiffHDR, which looks useful for making both captured & AI-generated vids more amenable to rich display and editing:

Most digital videos are stored in 8-bit low dynamic range (LDR) formats, where much of the original high dynamic range (HDR) scene radiance is lost due to saturation and quantization. This loss of highlight and shadow detail precludes mapping accurate luminance to HDR displays and limits meaningful re-exposure in post-production workflows. Although techniques have been proposed to convert LDR images to HDR through dynamic range expansion, they struggle to restore realistic detail in over- and underexposed regions.

To address this, we present DiffHDR, a framework that formulates LDR-to-HDR conversion as a generative radiance inpainting task in the latent space of a video diffusion model. By operating in Log-Gamma color space, DiffHDR leverages spatio-temporal generative priors from a pretrained video diffusion model to synthesize plausible HDR radiance in over- and underexposed regions while recovering the continuous scene radiance.

Our framework further enables controllable LDR-to-HDR video conversion guided by text prompts or reference images. To address the scarcity of paired HDR video data, we develop a pipeline that synthesizes high-quality HDR video training data from static HDRI maps. Extensive experiments demonstrate that DiffHDR significantly outperforms state-of-the-art approaches in radiance fidelity and temporal stability, producing realistic HDR videos with considerable latitude for re-exposure.

Tutorial: Use Astra to drive After Effects

After Effects, AI/ML

As I showed last week (i.e. ~3 million years ago in AI time!), it’s now possible to use AI to choreograph the creation of complex animations using After Effects:

Here’s a solid 8-minute tutorial on how to get up and running:

Summary courtesy of Gemini:

The three-step workflow:

  1. Storyboarding (1:50 – 4:05): Before building, use Astra to generate a storyboard. This ensures visual alignment and prevents inefficient prompting loops. You can also provide reference videos for Astra to analyze and use as stylistic inspiration.
  2. Prompting & Building (4:05 – 6:23): Provide a detailed prompt to Codex describing your desired scenes (beats) and effects. You can use provided templates from the creator’s GitHub repository. The AI will then generate scripts, perform the animation in After Effects, and automatically check its own work before presenting the final project.
  3. Editing (6:23 – 7:52): Finalize your project using follow-up prompts to refine specific graphics, transitions, or audio. The creator emphasizes the importance of human oversight, especially for managing transition speeds between scenes.

A little prototype expression-changer

AI/ML, User Interface

Cool work from Gabi on the Firefly team; see details in the tweet.

Ironically, Adobe shipped interactive expression control six years ago (!) in Neural Filters, and I left Google to return to work on that stuff at Adobe. The quality & generalizability of GANs let us down, however. It’s great to see continued progress now.

Go flat->layered with “Ad Delayer”

AI/ML, Design

I can’t say I grok the name, but whatever: not unlike Canva’s Magic Layers feature, Ad Delayer promises, “The ad you already paid for becomes a working source file again.”

Controlling a generative scene via Intangible 3D

3D, AI/ML

I’m a fan of these guys. Charles is awesome, and my friend Philip (with whom I got my start in his pre-ILM/Pixar days) is leading product design. I’m excited to try out their “semantic scene architecture,” which promises to focus on outcomes rather than inputs:

GPT Astra driving AE & Premiere Pro

After Effects, AI/ML

Desire to know more intensifies…

“Everyone in the world now has a 3D designer at their fingertips”

3D, AI/ML

Insane that 3D worlds can be conjured straight from language, then rendered natively or used as guidance for AI-based rendering. Check out this demo from my former Adobe teammate Tom:

Days of Miracles & Wonder, man—always.

Behind-the-scenes thread:

And yet more magic:

Photoshop combines generative imaging & masking

AI/ML, Photography

It seems I’m on something of a Photoshop kick lately (something something you can take the boy out of West 10, but…), so I’ll mention a couple of interesting-sounding enhancements that have landed in the latest release:

  • Instruct Edit with Masks, powered by Firefly Image 5, understands the full context of your image so you no longer need to precisely mark every edit area yourself. Just describe the precise edits you’re looking for — such as opening closed eyes or placing a hat on a specific person’s head — with simple prompts. The unmasked areas remain untouched so your other important elements, like faces, logos, or brand assets, are protected.
  • Light Adjustment Layer gives you professional-grade lighting controls — Exposure, Contrast, Highlights, Shadows, Whites, and Blacks — in a new non-destructive adjustment layer, so you get Camera Raw-level results without ever leaving your Photoshop layers workflow.
  • Markup lets you visually communicate edits by drawing directly on an image to show the model what you want — select areas to recolor, sketch arrows to indicate position, or brush in rough shapes to suggest new elements — so you can get results that more closely match your vision while reducing the ambiguity of text-only prompts.

Omni Flash: Poolside AI

AI/ML, Google Omni

See, I just take Gemini to the dog park, but my teammate Genevieve took it all the way to the pool on vacation. There are levels to this stuff. 🙂

But seriously: this is a really nice visualization of the new clip-extension feature, among other things. That’s how the scene can run a full 30 seconds.

Going inside “God’s Eye View”

3D

“It started as a slightly ridiculous question,” says Bilawal Sidhu: “How much of the planet can you make sense of from public data and a browser tab?

Being able to navigate & understand the world with a Jarvis-style UI is just magical.

Once aircraft, vessels, satellites, active fires, earthquakes, traffic, public cameras and physical infrastructure live on the same photorealistic 3D globe, it stops feeling like a pile of dashboards. It feels like a command center.

Amazingly, he’s made the whole thing open source. I just hope my man doesn’t get black-bagged by some dudes in a panel van. :-p

In this video I walk through the whole thing: ask the planet what’s happening over a cluster of military flights; jump into cockpit mode; voice-fly to points of interest, draw real boundaries and routes; move from city traffic into street cameras; inspect the physical internet; tune into radio around the world; and replay a recent Falcon 9 launch.

Higgsfield relighting looks wild

AI/ML, Relighting

I’ll say it again: Forget the video component per se: interactivity like this is either the future of Photoshop, or it’s the replacement for Photoshop. There’s no third way.

Gemini Omni 1.1 Flash is here! 4k, clip extension and more

AI/ML, Google Omni

Come build beautiful things with higher resolutions (and lower: 360p is great for fast, cheap drafts), clip extension (up to 10s at a time, up to 40s total), better support for audio and video references, improved frame interpolation (specifying first/last frames), and more. Let’s tell great stories together. 🙂

From the official blog:

  • More creative control with start and end frames. This helps keep the characters and narrative consistent as you thoughtfully transition between frames.
  • Elevate your rough cuts into polished pieces. Export crisp video in 1080p or 4K, ready for high-end digital, social, or broadcast editing workflows.
  • Draft videos quickly and upscale for quality. Test out concepts and compositions at a faster, lower-credit 360p resolution before committing to a full-resolution render. Once you have a clip you’re happy with, you can then download it in 720p resolution. This is particularly helpful in the Google Flow app — you can quickly draft videos on your phone on the go, and upscale your favorite versions.

See this thread for great examples, and please let me know if you have questions or requests. Onward!