Monthly Archives: December 2022

AI Snoop Dogg has arrived

AI/ML

…y’know, for all of you who were waiting. 🙄

I’m not sure what to say about “The first rap fully written and sung by an AI with the voice of Snoop Dogg,” except that now I really want the ability to drop in collaborations by other well known voices—e.g. Christopher Walken.

Maybe someone can now lip-sync it with the faces of YoDogg & friends:

Heinz AI Ketchup

AI/ML, Design

Life’s like a mayonnaise soda…
What good is seeing eye chocolate…

— Lou Reed

The marketers at Heinz had a little fun noticing that an AI image-making app (DALL•E, I’m guessing) tended to interpret requests for “ketchup” in the style of Heinz’s iconic bottle. Check it out:

ArtStation, Kickstarter, and others share their AI art policies

AI/ML

The whole community of creators, including toolmakers, continues to feel its way forward in the fast-moving world of AI-enabled image generation. For reference, here are some of the statements I’ve been seeing:

  • ArtStation has posted guidance on “Use of AI Software on ArtStation.”
    • Projects tagged using “NoAI” will automatically be assigned an HTML “NoAI” meta tag.
    • Projects won’t be assigned this tag automatically, as the site wants creators to choose whether or not their work is eligible for use in training.
  • Kickstarter has shared “Our Current Thinking on the Use of AI-Generated Image Software and AI Art.”
    • “Kickstarter must, and will always be, on the side of creative work and the humans behind that work. We’re here to help creative work thrive.”
    • Key questions they’ll ask include “Is a project copying or mimicking an artist’s work?” and “Does a project exploit a particular community or put anyone at risk of harm?”
  • From 3dtotal Publishing:
    • “3dtotal has four fundamental goals. One of them is to support and help the artistic community, so we cannot support AI art tools as we feel they hurt this community.”
  • Clip Studio Paint will no longer implement an image generator function:
    • “We received a lot of feedback from the community and will no longer implement the image generator palette.”
    • They “fear that this will make Clip Studio Paint artwork synonymous with AI-generated work” and are choosing to prioritize other features.
  • The Society of Illustrators has shared their thoughts:
    • “We oppose the commercial use of Artificially manufactured images and will not allow AI into our annual competitions at all levels.”
    • “AI was trained using copyrighted images. We will oppose any attempts to weaken copyright protections, as that is the cornerstone of the illustration community.”

More NeRF magic: From Michelangelo to NYC

3D, AI/ML, NeRF

This stuff—creating 3D neural models from simple video captures—continues to blow my mind. First up is Paul Trillo visiting the David:

Then here’s AJ from the NYT doing a neat day-to-night transition:

And lastly, Hugues Bruyère used a 360º camera to capture this scene, then animate it in post (see thread for interesting details):

https://twitter.com/smallfly/status/1604609303255605251?s=20&t=jdSW1NC_n54YTxsnkkFPJQ

“The Book of Leaves”

Photography

Obsessive (in a good way) photographer & animator Brett Foxwell has gathered & sequenced thousands of individual leaves into a mesmerizing sequence:

This is the complete leaf sequence used in the accompanying short film LeafPresser. While collecting leaves, I conceived that the leaf shape every single plant type I could find would fit somewhere into a continuous animated sequence of leaves if that sequence were expansive enough. If I didn’t have the perfect shape, it meant I just had to collect more leaves.

[Via]

A cool, quick demo of Midjourney->3D

3D, AI/ML

Numerous apps are promising pure text-to-geometry synthesis, as Luma AI shows here:

On a more immediately applicable front, though, artists are finding ways to create 3D (or at least “two-and-a-half-D”) imagery right from the output of apps like Midjourney. Here’s a quick demo using Blender:

In a semi-related vein, I used CapCut to animate a tongue-in-cheek self portrait from my friend Bilawal:

https://twitter.com/jnack/status/1599476677918478337?s=20&t=vu_Q7Wme3Q3Ueqp1WaGUpA

[Via Shi Yan]

Helping artists control whether AI trains on their work

AI/ML, Shit That Actually Matters

I believe strongly that creative tools must honor the wishes & rights of creative people. Hopefully that sounds thuddingly obvious, but it’s been less obvious how to get to a better state than the one we now inhabit, where a lot of folks are (quite reasonably, IMHO) up in arms about AI models having been trained on their work, without their consent. People broadly agree that we need solutions, but getting to them—especially via big companies—hasn’t been quick.

Thus it’s great to see folks like Mat Dryhurst & Holly Herndon driving things forward, working with Stability.ai and others to define opt-out/-in tools & get buy-in from model trainers. Check out the news:

https://twitter.com/spawning_/status/1603126330261897217

Here’s a concise explainer vid from Mat:

Adobe celebrates its 40th anniversary

Miscellaneous

It’s wild to look back & realize that I’ve spent roughly a third of my life at this special place, making amazing friends & even meeting my future wife (and future coworker!) on a customer visit. I feel like I should have more profundity to offer, and maybe I will soon, but at the moment I just feel grateful—including for the banger of a party the company threw last week in SF.

Here’s a fun little homage to history, made now via Photoshop 1.0. (I still kinda wish I hadn’t been talked into donating my boxed copy of 1.0 to the Smithsonian! The ‘Dobe giveth…)

PDF to cloud to homegrown tech titan: Adobe celebrates 40th anniversary

DALL•E/Stable Diffusion Photoshop panel gains new features

AI/ML, DALL•E

Our friend Christian Cantrell (20-year Adobe vet, now VP of Product at Stability.ai) continues his invaluable world to plug the world of generative imaging directly into Photoshop. Check out the latest, available for free here:

More NeRF magic: Dolly zoom & beyond

3D, AI/ML, NeRF

It’s insane to me how much these emerging tools democratize storytelling idioms—and then take them far beyond previous limits. Recently Karen X. Cheng & co. created some wild “drone” footage simply by capturing handheld footage with a smartphone:

Now they’re creating an amazing dolly zoom effect, again using just a phone. (Click through to the thread if you’d like details on how the footage was (very simply) captured.)

Meanwhile, here’s a deeper dive on NeRF and how it’s different from “traditional” photogrammetry (e.g. in capturing reflective surfaces):

Disney demos new aging/de-aging tech

AI/ML

Check out the latest magic, as described by Gizmodo:

To make an age-altering AI tool that was ready for the demands of Hollywood and flexible enough to work on moving footage or shots where an actor isn’t always looking directly at the camera, Disney’s researchers, as detailed in a recently published paper, first created a database of thousands of randomly generated synthetic faces. Existing machine learning aging tools were then used to age and de-age these thousands of non-existent test subjects, and those results were then used to train a new neural network called FRAN (face re-aging network).

When FRAN is fed an input headshot, instead of generating an altered headshot, it predicts what parts of the face would be altered by age, such as the addition or removal of wrinkles, and those results are then layered over the original face as an extra channel of added visual information. This approach accurately preserves the performer’s appearance and identity, even when their head is moving, when their face is looking around, or when the lighting conditions in a shot change over time. It also allows the AI generated changes to be adjusted and tweaked by an artist, which is an important part of VFX work: making the alterations perfectly blend back into a shot so the changes are invisible to an audience.

AI-made avatars for LinkedIn, Tinder, and more

AI/ML, Illustration

As I say, another day, another specialized application of algorithmic fine-tuning. Per Vice:

For $19, a service called PhotoAI will use 12-20 of your mediocre, poorly-lit selfies to generate a batch of fake photos specially tailored to the style or platform of your choosing. The results speak to an AI trend that seems to regularly jump the shark: A “LinkedIn” package will generate photos of you wearing a suit or business attire…

…while the “Tinder” setting promises to make you “the best you’ve ever looked”—which apparently means making you into an algorithmically beefed-up dudebro with sunglasses. 

Meanwhile, the quality of generated faces continues to improve at a blistering pace: