Enjoy some hypnotic work from Gregory Wieber:
Man, it is so weird to watch this scene with a laugh track & those whimsical mouth sounds:
Check out this eye-popping capture of the World Cup celebration in Buenos Aires, and see pilot Ale Petra’s Instagram feed for lots more FPV goodness:
If I had just upsetting amounts of disposable income, I might have to get this for my son. 🙃
Playground has gotten a jump on implementing some emerging research that enables making changes to an image simply by describing them:
In my very limited testing I’ve found the results to be a bit hit (“add a mustache”)…
…and miss (“give him a mohawk”) 😜:
Beautiful, haunting mutations, envisioned by Elastic:
Last month Paul Trillo shared some wild visualizations he made by walking around Michelangelo’s David, then synthesizing 3D NeRF data. Now he’s upped the ante with captures from the Louvre:
Over in Japan, Tommy Oshima used the tech to fly around, through, and somehow under a playground, recording footage via a DJI Osmo + iPhone:
I got my professional start at AGENCY.COM, a big dotcom-era startup co-founded by creative whirlwind Kyle Shannon. Kyle has been exploring AI imaging like mad, and recently he’s organized an AI Artists Salon that anyone is welcome to join in person (Denver) or online:
The AI Artists Salon is a collaborative group of creatively-minded people and we welcome anyone curious about the tsunami of inspiring generative technologies already rocking our our world. See Community Links & Resources.
On Tuesday evening I had the chance to present some ideas & progress that has inspired me—nothing confidential about Adobe work, of course, but hopefully illuminating nonetheless. If you’re interested, check it out (and pro tip: if you set playback to 1.5x speed or higher, I sound a lot sharper & funnier!).
Here’s an example made from a quick capture I did of my friend (nothing special, but amazing what one can get simply by walking in a circle while recording video):
Karen X. Cheng, back with another 3D/AI banger:
As luck (?) would have it, the commercial dropped on the third anniversary of my former teammate Jon Barron & collaborators bringing NeRFs into existence:
Thank God for the vibrant developer community—esp. Adobe vet Christian Cantrell (who somehow finds time to rev his plugin while serving as VP of product for Stability.ai):
— Christian Cantrell (@cantrell) January 14, 2023
“HEY MAN, you ever drop acid?? No? Well I do, and it looks *just like this*!!” — an excitable Googler when someone wallpapered a big meeting room in giant DeepDream renderings
In a similar vein, have fun tripping balls with AI, courtesy of Remi Molettee:
Bonus: Journey gets the treatment:
The company has announced a new mode for their Canvas painting app that turns simple brushstrokes into 360 environment maps for use in 3D apps or Omniverse. Check out this quick preview:
“I strongly believe that animation skills are going to be the next big thing in UI design,” says designer Michal Malewicz. Check out his full set of predictions for the year ahead:
The ongoing California storms have beaten the hell out of beloved little communities like Capitola, where the pier & cute seaside bungalows have gotten trashed. I found this effort by local artist Brighton Denevan rather moving:
PERSEVERE 💙 • 1-6-2023 • 3-5:30 • 8 Miles
The Santa Cruz Sentinel writes,
In the wake of the recent devastating storm damage to businesses in Capitola Village, local artist Brighton Denevan spent a few hours Friday on Capitola Beach sculpting the word “persevere” repeatedly in the sand to highlight a message of resilience and toughness that is a hallmark of our community. “The idea came spontaneously a few hours before low tide,” Denevan said. “After seeing all the destruction, it seemed like the right message for the moment.” Denevan has been drawing on paper since the age of 5 and picked up the rake and went out to the beach canvas in 2020 and each year I’ve done more projects. Last year, he created more than 200 works in the sand locally and across the globe.
Check out these gloriously detailed renderings from Markos Kay. I just wish the pacing were a little more chill so I could stare longer at each composition!
Kay has focused on the intersection of art and science in his practice, utilizing digital tools to visualize biological or primordial phenomena. “aBiogenesis” focuses a microscopic lens on imagined protocells, vesicles, and primordial foam that twists and oscillates in various forms.
My teammate CJ Gammon has released a handy new Chrome extension that lets you select any image, then use it as the seed for new image generation. Check it out:
In this beautiful work from Paul Trillo & co., AI extends—instead of replaces—human creativity & effort:
Here’s a peek behind the scenes:
This project would have never existed without the use of AI. A variety of tools were used from #dalle2 and #stablediffusion to generate the background assets Automatic1111 #img2img and @runwayml to process the video along with @AdobeAE to create the camera moves and transitions pic.twitter.com/FwqwWto966
— Paul Trillo (@paultrillo) December 29, 2022
Check out Karen X. Cheng’s clever use of simple wooden props + depth-to-image synthesis to create 3D renderings:
1. Take reference photo (you can use any photo – e.g. your real house, it doesn’t have to be dollhouse furniture)
2. Set up Stable Diffusion Depth-to-Image (google “Install Stable Diffusion Depth to Image YouTube”)
3. Upload your photo and then type in your prompts to remix the image
We recommend starting with simple prompts, and then progressively adding extra adjectives to get the desired look and feel. Using this method, @justinlv generated hundreds of options, and then we went through and cherrypicked our favorites for this video
Heh—I can’t quite say why I found this quick demo from developer & illustrator Marc Edwards both gripping & slightly nerve-racking, but his accuracy is amazing:
Check out this fun thread of Midjourney-made illustrations:
Hey friends—Happy New Year! I hope you’ve been able to get a little restful downtime, as I’ve done. I thought it’d be nice to ease back into things with these lovely titles from For All Mankind, which I’ve belatedly started watching & which I’m quite enjoying. The work is by Imaginary Forces, whom I’ve admired ever since seeing founder Kyle Cooper speak in the 90’s:
From the creators:
Lines deviate and converge in a graphic, tactile world that pays homage to the past while hinting at the “what if?” future explored throughout the series. Like the show logo itself, these lines weave and merge to create stylised representations of human exploration—badges, almost— ultimately reminding us of the common thread we share.
…y’know, for all of you who were waiting. 🙄
I’m not sure what to say about “The first rap fully written and sung by an AI with the voice of Snoop Dogg,” except that now I really want the ability to drop in collaborations by other well known voices—e.g. Christopher Walken.
Maybe someone can now lip-sync it with the faces of YoDogg & friends:
Heh—before the holidays get past us entirely, check out this novel approach to 3D motion capture from the always entertaining Kevin Parry:
[Via Victoria Nece]
Life’s like a mayonnaise soda…
What good is seeing eye chocolate…
— Lou Reed
The marketers at Heinz had a little fun noticing that an AI image-making app (DALL•E, I’m guessing) tended to interpret requests for “ketchup” in the style of Heinz’s iconic bottle. Check it out:
Whether or not you’re celebrating Christmas, I hope that you’re having a restful day & keeping warm with family & friends. Enjoy a couple of tidbits from the Nacks—including some Lego stop motion…
…and that heartwarming Christmas classic, Die Hard. 😅
Hah—enjoy this rather excellent mash-up:
For comparison, here are the original White Lotus titles:
The whole community of creators, including toolmakers, continues to feel its way forward in the fast-moving world of AI-enabled image generation. For reference, here are some of the statements I’ve been seeing:
- ArtStation has posted guidance on “Use of AI Software on ArtStation.”
- Projects tagged using “NoAI” will automatically be assigned an HTML “NoAI” meta tag.
- Projects won’t be assigned this tag automatically, as the site wants creators to choose whether or not their work is eligible for use in training.
- Kickstarter has shared “Our Current Thinking on the Use of AI-Generated Image Software and AI Art.”
- “Kickstarter must, and will always be, on the side of creative work and the humans behind that work. We’re here to help creative work thrive.”
- Key questions they’ll ask include “Is a project copying or mimicking an artist’s work?” and “Does a project exploit a particular community or put anyone at risk of harm?”
- From 3dtotal Publishing:
- “3dtotal has four fundamental goals. One of them is to support and help the artistic community, so we cannot support AI art tools as we feel they hurt this community.”
- Clip Studio Paint will no longer implement an image generator function:
- “We received a lot of feedback from the community and will no longer implement the image generator palette.”
- They “fear that this will make Clip Studio Paint artwork synonymous with AI-generated work” and are choosing to prioritize other features.
- The Society of Illustrators has shared their thoughts:
- “We oppose the commercial use of Artificially manufactured images and will not allow AI into our annual competitions at all levels.”
- “AI was trained using copyrighted images. We will oppose any attempts to weaken copyright protections, as that is the cornerstone of the illustration community.”
This stuff—creating 3D neural models from simple video captures—continues to blow my mind. First up is Paul Trillo visiting the David:
Then here’s AJ from the NYT doing a neat day-to-night transition:
And lastly, Hugues Bruyère used a 360º camera to capture this scene, then animate it in post (see thread for interesting details):
Obsessive (in a good way) photographer & animator Brett Foxwell has gathered & sequenced thousands of individual leaves into a mesmerizing sequence:
This is the complete leaf sequence used in the accompanying short film LeafPresser. While collecting leaves, I conceived that the leaf shape every single plant type I could find would fit somewhere into a continuous animated sequence of leaves if that sequence were expansive enough. If I didn’t have the perfect shape, it meant I just had to collect more leaves.
Numerous apps are promising pure text-to-geometry synthesis, as Luma AI shows here:
On a more immediately applicable front, though, artists are finding ways to create 3D (or at least “two-and-a-half-D”) imagery right from the output of apps like Midjourney. Here’s a quick demo using Blender:
In a semi-related vein, I used CapCut to animate a tongue-in-cheek self portrait from my friend Bilawal:
[Via Shi Yan]
I believe strongly that creative tools must honor the wishes & rights of creative people. Hopefully that sounds thuddingly obvious, but it’s been less obvious how to get to a better state than the one we now inhabit, where a lot of folks are (quite reasonably, IMHO) up in arms about AI models having been trained on their work, without their consent. People broadly agree that we need solutions, but getting to them—especially via big companies—hasn’t been quick.
Thus it’s great to see folks like Mat Dryhurst & Holly Herndon driving things forward, working with Stability.ai and others to define opt-out/-in tools & get buy-in from model trainers. Check out the news:
Here’s a concise explainer vid from Mat:
It’s wild to look back & realize that I’ve spent roughly a third of my life at this special place, making amazing friends & even meeting my future wife (and future coworker!) on a customer visit. I feel like I should have more profundity to offer, and maybe I will soon, but at the moment I just feel grateful—including for the banger of a party the company threw last week in SF.
Here’s a fun little homage to history, made now via Photoshop 1.0. (I still kinda wish I hadn’t been talked into donating my boxed copy of 1.0 to the Smithsonian! The ‘Dobe giveth…)
Raise your hand if you’re a Day 1.0 @Photoshop fan 🙋♀️
Nostalgia brought to you by TikTok’s wes45678 to celebrate Adobe’s 40th anniversary! pic.twitter.com/Y8QEtcJawf
— Adobe (@Adobe) December 12, 2022
Our friend Christian Cantrell (20-year Adobe vet, now VP of Product at Stability.ai) continues his invaluable world to plug the world of generative imaging directly into Photoshop. Check out the latest, available for free here:
“What if you could just focus on what’s being said, instead of how it sounds?” Sounds good to me. 🥁
It’s insane to me how much these emerging tools democratize storytelling idioms—and then take them far beyond previous limits. Recently Karen X. Cheng & co. created some wild “drone” footage simply by capturing handheld footage with a smartphone:
Now they’re creating an amazing dolly zoom effect, again using just a phone. (Click through to the thread if you’d like details on how the footage was (very simply) captured.)
Meanwhile, here’s a deeper dive on NeRF and how it’s different from “traditional” photogrammetry (e.g. in capturing reflective surfaces):
Check out the latest magic, as described by Gizmodo:
To make an age-altering AI tool that was ready for the demands of Hollywood and flexible enough to work on moving footage or shots where an actor isn’t always looking directly at the camera, Disney’s researchers, as detailed in a recently published paper, first created a database of thousands of randomly generated synthetic faces. Existing machine learning aging tools were then used to age and de-age these thousands of non-existent test subjects, and those results were then used to train a new neural network called FRAN (face re-aging network).
When FRAN is fed an input headshot, instead of generating an altered headshot, it predicts what parts of the face would be altered by age, such as the addition or removal of wrinkles, and those results are then layered over the original face as an extra channel of added visual information. This approach accurately preserves the performer’s appearance and identity, even when their head is moving, when their face is looking around, or when the lighting conditions in a shot change over time. It also allows the AI generated changes to be adjusted and tweaked by an artist, which is an important part of VFX work: making the alterations perfectly blend back into a shot so the changes are invisible to an audience.
As I say, another day, another specialized application of algorithmic fine-tuning. Per Vice:
For $19, a service called PhotoAI will use 12-20 of your mediocre, poorly-lit selfies to generate a batch of fake photos specially tailored to the style or platform of your choosing. The results speak to an AI trend that seems to regularly jump the shark: A “LinkedIn” package will generate photos of you wearing a suit or business attire…
…while the “Tinder” setting promises to make you “the best you’ve ever looked”—which apparently means making you into an algorithmically beefed-up dudebro with sunglasses.
Meanwhile, the quality of generated faces continues to improve at a blistering pace:
✨ Trained my own model for https://t.co/ll0YGEo53Z for more photorealistic renders called
I think by this week I can deploy it!
🤖 These are all 100% AI-generated people
Skin finally has pores now but don’t look at the hands yet please 😂 pic.twitter.com/Y6wbPz3BSS
— @levelsio (@levelsio) November 21, 2022
The Doggfather recently shared a picture of himself (rendered presumably via some Stable Diffusion/DreamBooth personalization instance)…
…thus inducing fans to reply with their own variations (click tweet above to see the thread). Among the many fun Snoop Doggs (or is it Snoops Dogg?), I’m partial to Cyberpunk…
Cyberpunk Snoop Dogg, 1,2,3 or 4? pic.twitter.com/w8BgeJBx86
— Techietree.eth/tez (@techietree_eth) November 29, 2022
— NIDO (@OfficialNID0) November 28, 2022
I meant to share this one last month, but there’s just no keeping up with the pace of progress!
My initial results are on the uncanny side, but more skillful practitioners like Paul Trillo have been putting the tech to impressive use:
Among the many, many things for which I can give thanks this year, I want to express my still-gobsmacked appreciation of the academic & developer communities that have brought us this year’s revolution in generative imaging. One of those developers is our friend & Adobe veteran Christian Cantrell, and he continues to integrate new tech from his new company (Stability AI) into Photoshop at a breakneck pace. Here’s the latest:
Here he provides a quick comparison between results from the previous Stable Diffusion inpainting model (top) & the latest one:
In any event, wherever you are & however you celebrate (or don’t), I hope you’re well. Thanks for reading, and I wish all the best for the coming year!
Among the great pleasures of this year’s revolutions in AI imaging has been the chance to discover & connect with myriad amazing artists & technologists. I’ve admired the work of Nathan Shipley, so I was delighted to connect him with my self-described “grand-mentee” Joanne Jang, PM for DALL•E. Nathan & his team collaborated with the Dalí Museum & OpenAI to launch Dream Tapestry, a collaborative realtime art-making experience.
The Dream Tapestry allows visitors to create original, realistic Dream Paintings from a text description. Then, it stitches a visitor’s Dream Painting together with five other visitors’ paintings, filling in the spaces between them to generate one collective Dream Tapestry. The result is an ever-growing series of entirely original Dream Tapestries, exhibited on the walls of the museum.
Check it out:
Another day, another special-purpose variant of AI image generation.
A couple of years ago, MyHeritage struck a chord with the world via Deep Nostalgia, an online app that could animate the faces of one’s long-lost ancestors. In reality it could animate just about any face in a photo, but I give them tons of credit for framing the tech in a really emotionally resonant way. It offered not a random capability, but rather a magical window into one’s roots.
Interestingly (perhaps only to me), it’s been hard for MyHeritage to sustain the kind of buzz generated by Deep Nostalgia. They later introduced the much more ambitious DeepStory, which lets you literally put words in your ancestors’ mouths. That seems not to have bent the overall needle in awareness, at least in the way that the earlier offering did. Let’s see how portrait generation fares.
Speaking of Bilawal, and in the vein of the PetPortrait.ai service I mentioned last week, here’s a fun little video in which he’s trained an AI model to create images of his mom’s dog. “Oreo lookin’ FESTIVE in that sweater, yo!” 🥰 I can only imagine that this kind of thing will become mainstream quickly.