With the update that starts rolling out today, you’ll see more videos — including the best snippets from your longer videos that Photos will automatically select and trim so you can relive the most meaningful moments. Even your still photos will feel more dynamic thanks to a subtle zoom that brings movement to your memories. And to bring it all together, next month we’ll start adding instrumental music to some Memories.
Happily, they’ve finally built a subset of the collage editor I spec’d out eight years ago (🧂🤷🏼).
Also,
Soon, you’ll begin to see full Cinematic Memories that transform multiple still photos into an end-to-end cinematic experience, taking you back to that moment in time. Cinematic Memories will also have music, making your photos feel a little more like a movie.
A quarter billion people engage with AR content every day, the company says.
And interestingly, one need not create a complex lens in order to have it pay off:
“The research found that simple AR can be just as performant as a sophisticated, custom Lens in driving both upper and lower-funnel metrics like brand awareness and purchase intent. Brands with the resources to execute a more sophisticated Lens will see additional benefits in mid-funnel brand metrics, including favorability and consideration.”
Well… kinda? I’m feeling somewhat hoodwinked, though. The new cam promises 72-megapixel captures, compared to 18 from its predecessor. This happens via some kind of 4x upsampling, it appears, and at least right now that’s incompatible with shooting HDR images.
Thus, as you can see via the comparisons below & via these original images, I was able to capture somewhat better detail (e.g. look at text) at the cost of getting worse tonal range (e.g. see the X2 lying on top of the book).
17 megapixel HDR at left; 72 megapixel normal at right
I need to carve out time to watch the tutorial below on how to wring the best out of this new cam.
The system uses images with descriptions to learn what the world looks like and how it is often described. It also uses unlabeled videos to learn how the world moves. With this data, Make-A-Video lets you bring your imagination to life by generating whimsical, one-of-a-kind videos with just a few words or lines of text.
Whew—no more wheedling my “grand-mentee” Joanne on behalf of colleagues wanting access. 😅
Starting today, we are removing the waitlist for the DALL·E beta so users can sign up and start using it immediately. More than 1.5M users are now actively creating over 2M images a day with DALL·E—from artists and creative directors to authors and architects—with over 100K users sharing their creations and feedback in our Discord community.
We are currently testing a DALL·E API with several customers and are excited to soon offer it more broadly to developers and businesses so they can build apps on this powerful system.
It’s hard to overstate just how much this groundbreaking technology has rocked our whole industry—all since publicly debuting less than 6 months ago! Congrats to the whole team. I can’t wait to see what they’re cooking up next.
Depending on how well it works, tech like this could be the greatest unlock in 3D creation the world has ever known.
The company blog post features interesting, promising details:
Though quicker than manual methods, prior 3D generative AI models were limited in the level of detail they could produce. Even recent inverse rendering methods can only generate 3D objects based on 2D images taken from various angles, requiring developers to build one 3D shape at a time.
GET3D can instead churn out some 20 shapes a second when running inference on a single NVIDIA GPU — working like a generative adversarial network for 2D images, while generating 3D objects. […]
GET3D gets its name from its ability to Generate Explicit Textured 3D meshes — meaning that the shapes it creates are in the form of a triangle mesh, like a papier-mâché model, covered with a textured material. This lets users easily import the objects into game engines, 3D modelers and film renderers — and edit them.
See also Dream Fields (mentioned previously) from Google:
The Corridor Crew has been banging on Stable Diffusion & Google’s new DreamBooth tech (see previous) that enables training the model to understand a specific concept—e.g. one person’s face. Here they’ve trained it using a few photos of team member Sam Gorski, then inserted him into various genres:
From there they trained up models for various guys at the shop, then created an illustrated fantasy narrative. Just totally incredible, and their sheer exuberance makes the making-of pretty entertaining:
I’m really excited to see this work from artists Holly Dryhurst & Mat Herndon. From Input Mag:
Dryhurst and Herndon are developing a standard they’re calling Source+, which is designed as a way of allowing artists to and opt into — or out of — allowing their work being used as training data for AI. (The standard will cover not just visual artists, but musicians and writers, too.) They hope that AI generator developers will recognize and respect the wishes of artists whose work could be used to train such generative tools.
Source+ (now in beta) is a product of the organization Spawning… [It] also developed Have I Been Trained, a site that lets artists see if their work is among the 5.8 billion images in the Laion-5b dataset, which is used to train the Stable Diffusion and MidJourney AI generators. The team plans to add more training datasets to pore through in the future.
The creators also draw a distinction between the rights of living vs. dead creators:
The project isn’t aimed at stopping people putting, say, “A McDonalds restaurant in the style of Rembrandt” into DALL-E and gazing on the wonder produced. “Rembrandt is dead,” Dryhurst says, “and Rembrandt, you could argue, is so canonized that his work has surpassed the threshold of extreme consequence in generating in their image.” He’s more concerned about AI image generators impinging on the rights of living, mid-career artists who have developed a distinctive style of their own.
And lastly,
“We’re not looking to build tools for DMCA takedowns and copyright hell,” he says. “That’s not what we’re going for, and I don’t even think that would work.”
On a personal note, I’m amused to see what the system thinks constitutes “John Nack”—apparently chubby German-ish old chaps…? 🙃
Great to see my old teammates (with whom I was working to enable cloud-rendered as well as locally rendered 3D experiences) continuing their work.
NASA and Google Arts & Culture have partnered to bring more than 60 3D models of planets, moons and NASA spacecraft to Google Search. When you use Google Search to learn about these topics, just click on the View in 3D button to understand the different elements of what you’re looking at even better. These 3D annotations will also be available for cells, biological concepts (like skeletal systems), and other educational models on Search.
The makers of this new search engine say they’re already serving more than 200,000 images/day & growing rapidly. Per this article, “It’s a massive collection of over 5 million Stable Diffusion images including its text prompts.” Just get ready to see some… interesting art (?). 🙃
'Consonance' is my project that explores how AI interprets the spoken word. This is an excerpt from a James Joyce novel. I use his words exactly for the prompt, in the style of artist John Lavery. Funded by @futurescreensni A collaboration with @HeaneyCentre + Armchair & Rocket pic.twitter.com/8sGgipjZeb
Karen X. Cheng & pals (including my friend August Kamp) went to work extending famous works by Vermeer, Da Vinci, and Magritte, then placing them into AR filter (which you can launch from the post) that lets you walk right into the scenes. Wild!
Who’s got two thumbs & just pulled the trigger? This guuuuuy. 😌
Now, will it be worth it? I sure hope so.
Fortunately I got to try out the much larger & more expensive One R 1″ Edition back in July & concluded that it’s not for me (heavier, lacking Bullet Time, and not producing appreciably better quality results—at least for the kind of things I shoot).
I’m of course hoping the X3 (success to my much-beloved One X2) will be more up my alley. Here’s some third-party perspective:
“Shoon is a recently released side scrolling shmup,” says Vice, “that is fairly unremarkable, except for one quirk: it’s made entirely with art created by Midjourney, an AI system that generates images from text prompts written by users.’ Check out the results:
Magdalena Bay has shared a new Felix Geen directed video for “Dreamcatching.” The clip, multi-dimensional explored through cutting-edge AI technology and GAN artwork, combined with VQGAN+CLIP, is a technique that utilizes a collection of neural networks that work in unison to generate images based on input text and/or images.
Creative director Wes Phelan shared this charming little summary of how he creates kids’ books & games using DALL•E, including their newly launched outpainting support:
Let the canvases extend in every direction! The thoughtfully designed new tiling UI makes it easy to synthesize adjacent chunks in sequence, partly overcoming current resolution limits in generative imaging:
We just released a new edit interface for DALL·E that lets you use Outpainting to expand beyond the original borders of an image!
You can use this to make images with different aspect ratios, or arbitrarily large images like murals or magazine covers. pic.twitter.com/OW4lC6HQFl
Speaking of Paul here’s a fun new little VFX creation made using DALL•E:
AI is going to change VFX. This is a silly little experiment but it shows how powerful dall-e 2 is in generating elements into a pre existing video. These tools will become easier to use so when spectacle becomes cheap, ideas will prevail#aiart#dalle#ufo@openaidalle#dalle2pic.twitter.com/XGHy9uY09H
I… I just can’t handle it: this tech is advancing so fast, my hair is whipping back. 😅
My old teammate Yael Pritch & team have announced DreamBooth: by providing 3-5 images of a subject, you can fine-tune a model of that subject, then generate variations (e.g. changing the environment and context).
“Everything Is A Remix,” right? Or, if you prefer, it’s an homage—as Star Wars arguably was to The Dambusters. Anyway, genius steals, blah blah, and this is a fun set of juxtapositions & tropes:
The latest beta build of Photoshop contains a new feature called Photo Restoration. Whenever I have seen new updates in AI photo restoration over the last few years, I have tried the technology on an old family photo that I have of my great great great grandfather. A Scotsman who lived between 1845-1919. I applied the neural filter plus colorize technique to update the image in Photoshop. The restored photo is on the left, the original on the right. It is really astonishing how advanced AI is becoming.
Learn more about accessing the feature in Photoshop here.
This little guy looks like a kick in the pants to fly, and “turtle mode”—which gives the drone the ability to flip itself over & fly again after a collision—seems really smart:
We ask: how can we use language-guided models to turn our cat into a painting, or imagine a new product based on our favorite toy? Here we present a simple approach that allows such creative freedom.
Using only 3-5 images of a user-provided concept, like an object or a style, we learn to represent it through new “words” in the embedding space of a frozen text-to-image model. These “words” can be composed into natural language sentences, guiding personalized creation in an intuitive way.
The new open-source Stable Diffusion model is pretty darn compelling. Per PetaPixel:
“Just telling the AI something like ‘landscape photography by Marc Adamus, Glacial lake, sunset, dramatic lighting, mountains, clouds, beautiful’ gives instant pleasant looking photography-like images. It is incredible that technology has got to this point where mere words produce such wonderful images (please check the Facebook group for more).” — photographer Aurel Manea
Many years ago (nearly 10!), when I was in the thick of making up bedtime stories every night, I wished aloud for an app that would help do the following:
Record you telling your kids bedtime stories (maybe after prompting you just before bedtime)
Transcribe the text
Organize the sound & text files (into a book, journal, and/or timeline layout)
Add photos, illustrations, and links.
Share from the journal to a blog, Tumblr, etc.
I was never in a position to build it, but seeing this fusion of kid art + AI makes me hope again:
With #stablediffusion img2img, I can help bring my 4yr old’s sketches to life.
Elsewhere, After Effects ninja Paul Trillo is back at it with some amazing video-meets-DALL•E-inpainting work:
AI fashion show using dall-e to generate hundreds of outfits of the day after tomorrow. An interesting way to brainstorm costume and fashion design ideas.
What the what? From Sébastien Deguy, founder of Allegorithmic & now VP 3D & Immersive at Adobe:
I don’t communicate often about that other part of my activities, but I am so glad I could work with one of my all time favorites that I have to 🙂 My latest track, featuring Raekwon (Wu-Tang Clan) is available on all streaming platforms! Like here on Spotify.
Share your knowledge, passion and experience of Adobe Premiere Pro and After Effects with video makers.
Engage daily with communities around professional video wherever they are (Reddit, Twitter, Facebook, Instagram, etc.) in two-way conversations, representing Adobe.
Be active and visible within the community, build long term relationships and trust, and demonstrate that knowledge and understanding of the users to help Adobe internally.
Build relationships with leaders in the identified communities.
Establish yourself as a leader through your work and participation in time-sensitive topics and conversations.
Answer questions and engage in discussion about Adobe products and policies with a heavy focus on newcomers to the ecosystem.
Encourage others through sharing your personal use of and experimentation with professional video tools.
Enable conversation, build content and speak about Adobe tools to address the specific audience needs.
Understand the competitive landscape and promptly report accordingly.
Coordinate with other community, product, marketing and campaign teams to develop mini-engagements and activities for the community (i.e. AMAs with the product team on Reddit, or community activities and discussions via live streams, etc.)
Work closely with the broader community team, evangelism team, and product teams to provide insight and feedback to advocate for the pro video community within Adobe and to help drive product development direction.
We are teetering on the cusp of a Cambrian explosion in UI creativity, with hundreds of developers competing to put amazing controls atop a phalanx of ever-improving generative models. These next couple of months & years are gonna be wiiiiiiild.
After seeing years & years of AR demos featuring the placement of furniture, I once heard someone say in exasperation, “Bro… how much furniture do you think I buy?”
Happily here’s a decidedly fresh approach, surrounding the user & some real-world furniture with a projection of the person’s 3D-scanned home. Wild!
Now, how easy can 3D home scanning be made—and how much do people care about this kind of scenario? I don’t know, but I love what the tech can enable already.
What if you could bring your home shopping?
Our latest prototype lets you teleport real furniture from the store into a 3D scan of your living room. It's amazing to experience first hand. 🤩 pic.twitter.com/gAPawRXgGB
Watching this clip from the Today Show introduction of Photoshop in 1990, it’s amazing to hear the same ethical questions 32 years ago that we contend with now around AI-generated imagery. Also amazing: I now work with Russell‘s son Davis (our designer) to explore AI imaging + Photoshop and beyond.
The Today Show in 1990 talking about this new thing called Photoshop and how it will change photography and media.
Now the somewhat scarily named CLIP Interrogator promises exactly that kind of insight:
What do the different OpenAI CLIP models see in an image? What might be a good text prompt to create similar images using CLIP guided diffusion or another text to image model? The CLIP Interrogator is here to get you answers!
Here’s hoping it helps us get some interesting image -> text -> image flywheels spinning.
Hmm—this is no doubt brilliant tech, and I’d like to learn more, but I wonder about the Venn diagram between “Objects that people want in 3D,” “Objects for which a sufficiently large number of good images exist,” and “Objects for which good human-made 3D models don’t already exist.” In my experience photogrammetry is most relevant for making models from extremely specific subjects (e.g. a particular apartment) rather than from common objects that are likely to exist on Sketchfab et al. It’s entirely possible I’m missing a nuanced application here, though. As I say, cool tech!
I wish I’d gotten to work more with Steve Seitz at Google, as I’ve long admired his wide-ranging work (from Photosynth to Face Movies to the company’s new 3D video collaboration tech). Here he provides a pretty accessible overview of how large language models (e.g. those behind DALL•E & similar systems) actually work:
