In my time at Google, we tried and failed a lot to make virtual try-on happen using AR. It’s extremely hard to…

• measure bodies (to make buying decisions based on fit)
• render virtual clothing accurately (placing virtual clothing over real clothing, or getting them to disrobe, which is even harder!; simulating materials in realtime)
• get a sizable corpus of 3D assets (in a high-volume, low-margin industry)

Outside of a few limited pockets (trying on makeup, glasses, and shoes—all for style, not for fit), I haven’t seen anyone (Amazon, Snap, etc.) crack the code here. Researcher Ira Kemelmacher-Shlizerman (who last I heard was working on virtual mirrors, possibly leveraging Google’s Stargate tech) acknowledges this:

Current techniques like geometric warping can cut-and-paste and then deform a clothing image to fit a silhouette. Even so, the final images never quite hit the mark: Clothes don’t realistically adapt to the body, and they have visual defects like misplaced folds that make garments look misshapen and unnatural.

So, it’s interesting to see Google trying again (“Try on clothes with generative AI”):

This week we introduced an AI-powered virtual try-on feature that uses the Google Shopping Graph to show you how clothing will look on a diverse set of real models.

Our new guided refinements can help U.S. shoppers fine-tune products until you find the perfect piece. Thanks to machine learning and new visual matching algorithms, you can refine using inputs like color, style and pattern.

They’ve posted a technical overview and a link to their project site:

Inspired by Imagen, we decided to tackle VTO using diffusion — but with a twist. Instead of using text as input during diffusion, we use a pair of images: one of a garment and another of a person. Each image is sent to its own neural network (a U-net) and shares information with each other in a process called “cross-attention” to generate the output: a photorealistic image of the person wearing the garment. This combination of image-based diffusion and cross-attention make up our new AI model.

They note that “We don’t promise fit and for now focus only on visualization of the try on. Finally, this work focused on upper body clothing.”

It’s a bit hard to find exactly where one can try out the experience. They write:

Starting today, U.S. shoppers can virtually try on women’s tops from brands across Google, including Anthropologie, Everlane, H&M and LOFT. Just tap products with the “Try On” badge on Search and select the model that resonates most with you.

Paul is continuing to explore what’s possible by generating short clips using Runway’s Gen-2 text-to-video model. Check out the melancholy existential musings of “Thank You For Not Answering”:

And then, to entirely clear your mental palate, there’s the just deeply insane TRUCX!

I love this kind of silliness—just the right mix of idiom & constraint to bring out folks’ creativity.

Break-A-Scene seems incredibly cool, promising to extract objects from scenes, then remix them into other scenes while keeping them editable and preserving their appearances. Check out the 2-minute overview video:

The folks at Curious Refuge are back on their bullshit, in the best sense, dropping a new AI-powered Wes Anderson homage:

They’ve also launching a new AI filmmaking course:

Welcome to AI Filmmaking from Curious Refuge. This is the world’s first online course for showing you how to use AI to create films. Our training will cover various aspects of the production process from prompt engineering to animation and movement. We’d love for you to join our course and unlock your inner artist. $499 $399 Per Artist – Pre-Sale Special

“AI knows how things look, but not how they work.” Check out this really accessible overview of such a devilishly weird, funny phenomenon in AI art:

Yes, there is a day coming where I’ll tire of these AI-powered mashups.
No, that day is not today. 😛 (“Wow!”)

“In the quaintest corner of Middle Earth…” 😅 More awesomeness from Curious Refuge, who promise to share more details about their process soon.

Just a fun little moment in last week’s conversation with Deke, featuring multiple self-censored F bombs of pure wonderment. 😌

Longtime Adobe vet Christian Cantrell continues to build out his Concept.art startup while extending Photoshop via GPT and generative imagining. I can’t keep up with his daily progress on Twitter (recommendation: just go follow him there!), but check out some quick recent demos:

Telling Photoshop to lens flare all the things:

Adding blurs:

Removing backgrounds:

And finally, tying it all together:

Pyroclastic Horror IPA looks 🔥!

Now this is exactly the kind of thing I want to help bring into the world—not just because it’s delightful unto itself, but because it shows how AI-enabled tools can make the impossible possible, rather than displacing or diminishing artists’ work. It’s not like in some earlier world a talented team would’ve made this all by hand: 99% likely, it simply wouldn’t exist at all.

The 1% exception is exemplified by SNL’s brilliant Anderson parody from a few years back—all written, scouted, shot, and edited in ~3 days, but all requiring the intensive efforts of an incredibly skilled crew. (Oh, and it too features a terrific Owen Wilson spoof.)

My Adobe Research teammates & their collaborators at Berkeley have devised an interesting way to represent objects in scenes—not via sharply defined segments, but via more diffuse blobs. This enables some trippy editing techniques and results. Check it out in action:

The simple capture-and-augment experience promises “The magic of Gen-1. Now on your phone. Download the new Runway iOS app today.”

This is pretty great. 🔴🤖😛