Monthly Archives: October 2022

“Mundane Halloween” win: “Person whose skeleton is being estimated by machine learning” 

Happy day to all who celebrate. 😌

The whole thread is hilarious & well worth a look:

A fistful of generative imaging news

Man, I can’t keep up with this stuff—and that’s a great problem to have. Here are some interesting finds from just the last few days:

Adobe “Made In The Shade” sneak is 😎

OMG—interactive 3D shadow casting in 2D photos FTW! 🔥

In this sneak, we re-imagine what image editing would look like if we used Adobe Sensei-powered technologies to understand the 3D space of a scene – the geometry of a road and the car on the road, and the trees surrounding, the lighting coming from the sun and the sky, the interactions between all these objects leading to occlusions and shadows – from a single 2D photograph.

Check out AI backdrop generation, right in the Photoshop beta today

One of the sleeper features that debuted at Adobe MAX is the new Create Background, found under Neural Filters. (Note that you need to be running the current public beta release of Photoshop, available via the Creative Cloud app—y’know, that little “Cc” icon dealio you ignore in your menu bar. 🙃)

As this quick vid demonstrates, the filter can not only generate backgrounds based on text, it links to a Behance gallery containing images and popular prompts. You can use these visuals as inspiration, then use the prompts to produce artwork within the plugin:

https://youtu.be/oMVfxyQbO5c?t=74

Here’s the Behance browser:

New Lightroom features: A 1-minute tour, plus a glimpse of the future

The Lightroom team has rolled out a ton of new functionality, from smarter selections to adaptive presets to performance improvements. You should read up on the whole shebang—but for a top-level look, spend a minute with Ben Warde:

And looking a bit more to the future, here’s a glimpse at how generative imaging (in the style of DALL•E, Stable Diffusion, et al) might come into LR. Feedback & ideas welcome!

Stable Diffusion + Adobe Fonts = 🧙‍♂️🔥

Check out my teammates’ new explorations, demoed here on Adobe Express:

https://twitter.com/jnack/status/1582818166698217472?s=20&t=FYpJnxsd4aSiQixheZ9_SQ

Per the blog post:

Generative AI incorporated into Adobe Express will help less experienced creators achieve their unique goals. Rather than having to find a pre-made template to start a project with, Express users could generate a template through a prompt, and use Generative AI to add an object to the scene, or create a unique font based on their description. But they still will have full control — they can use all of the Adobe Express tools for editing images, changing colors, and adding fonts to create the flyer, poster, or social media post they imagine.

Turn images into usable Stable Diffusion prompts

LatentSpace.dev promises to turn your images into text prompts that can be used in Stable Diffusion to create new artwork. Watch it work:

It interpreted a pic of my old whip as being, among other things, a “5. 1975 pontiac firebird shooting brake wagon estate.” Not entirely bad! 😌

“Imagic”: Text-based editing of photos

It seems almost too good to be true, but Google Researchers & their university collaborators have unveiled a way to edit images using just text:

In this paper we demonstrate, for the very first time, the ability to apply complex (e.g., non-rigid) text-guided semantic edits to a single real image. For example, we can change the posture and composition of one or multiple objects inside an image, while preserving its original characteristics. Our method can make a standing dog sit down or jump, cause a bird to spread its wings, etc. — each within its single high-resolution natural image provided by the user.

Contrary to previous work, our proposed method requires only a single input image and a target text (the desired edit). It operates on real images, and does not require any additional inputs (such as image masks or additional views of the object).

I can’t wait to see it in action!

Stable Diffusion meets WebAR

Back at the start of my DALL•E journey, I wished aloud for a diffusion-powered mobile app:

https://twitter.com/jnack/status/1529977613623496704?s=20&t=dlYc1z2m-Cxb61G0KCaiIw

Now, thanks to the openness of Stable Diffusion & WebAR, creators are bringing that vision closer to reality:

https://twitter.com/stspanho/status/1581707753747537920?s=20&t=JPLmD_bV0U4Gkv2-2bJX-g

I can’t wait to see what’s next!

Blender + Stable Diffusion = 🪄

Easy placement/movement of 3D primitives -> realistic/illustrative rendering has long struck me as extremely promising. Using tech like StyleGAN to render from 3D can produce interesting results, but it’s been difficult to bring the level of quality & consistency up to what Adobe users demand.

Now with Stable Diffusion (and, one hopes, other diffusion models in the future) attached to Blender (and, one hopes, other object manipulation tools), the vision is getting closer to reality:

Wayback machine: When “AI” was “Adobe Illustrator”

Check out a fun historical find from Adobe evangelist Paul Trani:

The video below shipped on VHS with the very first version of Adobe Illustrator. Adobe CEO & Illustrator developer John Warnock demonstrated the new product in a single one-hour take. He was certainly qualified, being one of the four developers whose names were listed on the splash screen!

How lucky it was for the world that a brilliant graphics engineer (John) married a graphic designer (Marva Warnock) who could provide constant input as this groundbreaking app took shape. 

If you’re interested in more of the app’s rich history, check out The Adobe Illustrator Story:

Check out NeRF Studio & some eye-popping results

The power & immersiveness of rendering 3D from images is growing at an extraordinary rate. NeRF Studio promises to make creation much more approachable:

The kind of results one can generate from just a series of photos or video frames is truly bonkers:

Here’s a tutorial on how to use it:

Zooming around the world through Google Street View

“My whole life has been one long ultraviolent hyperkinetic nightmare,” wrote Mark Leyner in “Et Tu, Babe?” That thought comes to mind when glimpsing this short film by Adam Chitayat, stitched together from thousands of Street View images (see Vimeo page for a list of locations).

I love the idea—indeed, back in 2014 I tried to get Google Photos to stitch together visual segues that could interconnect one’s photos—but the pacing here has my old man brain pulling the e-brake after just some short exposure. YMMV, so here ya go:

[Via]

Google Photos redesigns Memories

Nice work from my old crew:

With the update that starts rolling out today, you’ll see more videos — including the best snippets from your longer videos that Photos will automatically select and trim so you can relive the most meaningful moments. Even your still photos will feel more dynamic thanks to a subtle zoom that brings movement to your memories. And to bring it all together, next month we’ll start adding instrumental music to some Memories.

Happily, they’ve finally built a subset of the collage editor I spec’d out eight years ago (🧂🤷🏼).

Also,

Soon, you’ll begin to see full Cinematic Memories that transform multiple still photos into an end-to-end cinematic experience, taking you back to that moment in time. Cinematic Memories will also have music, making your photos feel a little more like a movie.

Snapchat: Even simple AR is effective AR

A quarter billion people engage with AR content every day, the company says.

And interestingly, one need not create a complex lens in order to have it pay off:

“The research found that simple AR can be just as performant as a sophisticated, custom Lens in driving both upper and lower-funnel metrics like brand awareness and purchase intent. Brands with the resources to execute a more sophisticated Lens will see additional benefits in mid-funnel brand metrics, including favorability and consideration.”

Are high-res shots from the Insta360 X3 any good?

Well… kinda? I’m feeling somewhat hoodwinked, though. The new cam promises 72-megapixel captures, compared to 18 from its predecessor. This happens via some kind of 4x upsampling, it appears, and at least right now that’s incompatible with shooting HDR images.

Thus, as you can see via the comparisons below & via these original images, I was able to capture somewhat better detail (e.g. look at text) at the cost of getting worse tonal range (e.g. see the X2 lying on top of the book).

I need to carve out time to watch the tutorial below on how to wring the best out of this new cam.