“Imagic”: Text-based editing of photos

AI/ML, Photography

It seems almost too good to be true, but Google Researchers & their university collaborators have unveiled a way to edit images using just text:

In this paper we demonstrate, for the very first time, the ability to apply complex (e.g., non-rigid) text-guided semantic edits to a single real image. For example, we can change the posture and composition of one or multiple objects inside an image, while preserving its original characteristics. Our method can make a standing dog sit down or jump, cause a bird to spread its wings, etc. — each within its single high-resolution natural image provided by the user.

Contrary to previous work, our proposed method requires only a single input image and a target text (the desired edit). It operates on real images, and does not require any additional inputs (such as image masks or additional views of the object).

I can’t wait to see it in action!

Stable Diffusion meets WebAR

AI/ML, AR/VR

Back at the start of my DALL•E journey, I wished aloud for a diffusion-powered mobile app:

https://twitter.com/jnack/status/1529977613623496704?s=20&t=dlYc1z2m-Cxb61G0KCaiIw

Now, thanks to the openness of Stable Diffusion & WebAR, creators are bringing that vision closer to reality:

https://twitter.com/stspanho/status/1581707753747537920?s=20&t=JPLmD_bV0U4Gkv2-2bJX-g

I can’t wait to see what’s next!

Blender + Stable Diffusion = 🪄

3D, AI/ML

Easy placement/movement of 3D primitives -> realistic/illustrative rendering has long struck me as extremely promising. Using tech like StyleGAN to render from 3D can produce interesting results, but it’s been difficult to bring the level of quality & consistency up to what Adobe users demand.

Now with Stable Diffusion (and, one hopes, other diffusion models in the future) attached to Blender (and, one hopes, other object manipulation tools), the vision is getting closer to reality:

Wayback machine: When “AI” was “Adobe Illustrator”

Illustration

Check out a fun historical find from Adobe evangelist Paul Trani:

https://twitter.com/paultrani/status/1581008882541133824?s=46&t=XjcRX5DdV1OKyzGKVimjTA

The video below shipped on VHS with the very first version of Adobe Illustrator. Adobe CEO & Illustrator developer John Warnock demonstrated the new product in a single one-hour take. He was certainly qualified, being one of the four developers whose names were listed on the splash screen!

How lucky it was for the world that a brilliant graphics engineer (John) married a graphic designer (Marva Warnock) who could provide constant input as this groundbreaking app took shape. 

If you’re interested in more of the app’s rich history, check out The Adobe Illustrator Story:

Check out NeRF Studio & some eye-popping results

3D, AI/ML, NeRF

The power & immersiveness of rendering 3D from images is growing at an extraordinary rate. NeRF Studio promises to make creation much more approachable:

https://twitter.com/akanazawa/status/1577686321119645696?s=20&t=OA61aUUy3A6P1aMQiUIzbA

The kind of results one can generate from just a series of photos or video frames is truly bonkers:

Here’s a tutorial on how to use it:

Zooming around the world through Google Street View

Photography

“My whole life has been one long ultraviolent hyperkinetic nightmare,” wrote Mark Leyner in “Et Tu, Babe?” That thought comes to mind when glimpsing this short film by Adam Chitayat, stitched together from thousands of Street View images (see Vimeo page for a list of locations).

I love the idea—indeed, back in 2014 I tried to get Google Photos to stitch together visual segues that could interconnect one’s photos—but the pacing here has my old man brain pulling the e-brake after just some short exposure. YMMV, so here ya go:

[Via]

Google Photos redesigns Memories

Google Photos

Nice work from my old crew:

With the update that starts rolling out today, you’ll see more videos — including the best snippets from your longer videos that Photos will automatically select and trim so you can relive the most meaningful moments. Even your still photos will feel more dynamic thanks to a subtle zoom that brings movement to your memories. And to bring it all together, next month we’ll start adding instrumental music to some Memories.

Happily, they’ve finally built a subset of the collage editor I spec’d out eight years ago (🧂🤷🏼).

Also,

Soon, you’ll begin to see full Cinematic Memories that transform multiple still photos into an end-to-end cinematic experience, taking you back to that moment in time. Cinematic Memories will also have music, making your photos feel a little more like a movie.

Snapchat: Even simple AR is effective AR

AR/VR

A quarter billion people engage with AR content every day, the company says.

And interestingly, one need not create a complex lens in order to have it pay off:

“The research found that simple AR can be just as performant as a sophisticated, custom Lens in driving both upper and lower-funnel metrics like brand awareness and purchase intent. Brands with the resources to execute a more sophisticated Lens will see additional benefits in mid-funnel brand metrics, including favorability and consideration.”