Well gang, Imma say about this year what Dave Chapelle said at the Playa Haters Ball:
What can I say about that suit that hasn’t already been said about Afghanistan? It looks bombed out and depleted.
I’ll leave it to John Oliver to take it from here:
Here’s to a brighter, stronger, healthier year together, friends. God bless & stay strong.
Stay cozy, friends.
Wherever you are & whatever you may celebrate, I hope you and yours are well this Christmas. I’ve been amusing the fam by dialing up some old Chieftains Christmas tracks (which transport me back to my relatives’ cozy homes in Chicago growing up), including this absolute banger. 😌 Enjoy.
Although we can’t travel far this holiday season (bye bye, Mendocino! see you… someday?), we are, in the words of Tony Stark, “bringing the party to you” via our decked-out VW Westy (pics). I’m having fun experimenting with my new Insta360 One X2, mounting it on a selfie stick & playing with timelapse mode. Here’s a taste of the psychedelic stylings, courtesy of The Weeknd…
…and Brian Wilson:
Apropos of nothing, check out 60 lovingly rendered seconds commissioned by YouTube:
Maciej Kuciara writes,
MECHA – the love letter to our youth. Watching anime classics as kids left a mark that stayed with us to this day. So we felt it’s due to time to celebrate our love to mecha and pay proper homage with this piece we did for YouTube.
Jumpei Mitsui‘s work is staggering. Colossal writes,
During the course of 400 hours, Mitsui snapped together 50,000 cobalt and white LEGO into an undulating wave that mimics the original woodblock print.
To recreate this iconic work in three-dimensions, Mitsui studied videos of waves crashing and pored over academic papers on the topic. He then sketched a detailed model before assembling the textured water, three boats, and Mount Fuji that span more than five feet.
A couple of my old Adobe pals (who happen to dwell in the dark, beautiful wilderness around Santa Cruz) have been sharing some great astrophotography-related work lately.
First, Bryan O’Neil Hughes shares tips on photographing the heavens, including the Jupiter-Saturn Conjunction and the Ursids Meteor Shower:
Meanwhile Michael Lewis has been capturing heavenly shots which, in the words of my then ~4-year-old son, “make my mind blow away.” Check out his Instagram feed for images like this:
And if you’re shooting with a phone—especially with a Pixel—check out these tips from former Pixel imagining engineer Florian Kainz (who’s now also at Adobe—hat trick!).
Putting the “AR” in “Galaxy far, far away…” 😌
Just don’t expect to get a Baby Yoda Hellraiser Pinhead version. 😬
Also, remember that if you have a Pixel 5G or a compatible Android 5G device, you can install “The Mandalorian” AR Experience.
#ThisIsTheWay
I’m delighted to be closing out 2020 on a pair of high notes, welcoming the arrival of my two biggest efforts from the last year+.
First, Google Search now supports 150+ new cars that you can view in 3D and AR (via iPhone or Android device), including in beautiful cloud-rendered quality (provided you have a good connection & up-to-date Android). As we initially previewed in October:
Bring the showroom to you with AR
You can easily check out what the car looks like in different colors, zoom in to see intricate details like buttons on the dashboard, view it against beautiful backdrops and even see it in your driveway. We’re experimenting with this feature in the U.S. and working with top auto brands, such as Volvo and Porsche, to bring these experiences to you soon.
Second, you can try on AR beauty products right through Search:
Now, when you search for a lipstick or eyeshadow product, like L’Oreal’s Infallible Paints Metallic Eyeshadow, you can see what it looks like on a range of skin tones and compare shades and textures to help you find the right products.
To help you find the perfect match, you can now also virtually try makeup products right from the Google app.
Google researchers Ira Kemelmacher-Shlizerman, Brian Curless, and Steve Seitz have been working with University of Washington folks on tech that promises “30fps in 4K resolution, and 60fps for HD on a modern GPU.”
Our technique is based on background matting, where an additional frame of the background is captured and used in recovering the alpha matte and the foreground layer.
Check it out:
See the PDF, abstract, and GitHub repo for details.
Nearly 20 years ago, on one of my first customer visits as a Photoshop PM, I got to watch artists use PS + After Effects to extract people from photo backgrounds, then animate the results. The resulting film—The Kid Stays In The Picture—lent its name to the distinctive effect (see previous).
Now I’m delighted that Google Photos is rolling out similar output to its billion+ users, without requiring any effort or tools:
We use machine learning to predict an image’s depth and produce a 3D representation of the scene—even if the original image doesn’t include depth information from the camera. Then we animate a virtual camera for a smooth panning effect—just like out of the movies.
Photos is also rolling out new collages, like this:
And they’re introducing new themes in the stories-style Memories section up top as well:
Now you’ll see Memories surface photos of the most important people in your life… And starting soon, you’ll also see Memories about your favorite things—like sunsets—and activities—like baking or hiking—based on the photos you upload.
Enjoy!
I love these simple, practical uses of augmented reality. The Maps team writes,
Last month, we launched Live View in Location Sharing for Pixel users, and we’ll soon expand this to all Android and iOS users around the globe. When a friend has chosen to share their location with you, you can easily tap on their icon and then on Live View to see where and how far away they are–with overlaid arrows and directions that help you know where to go.
Live View in Location Sharing will soon expand to all Android and iOS users globally on ARCore and ARKit supported phones.
They’re also working hard to leverage visual data & provide better localization and annotation.
With the help of machine learning and our understanding of the world’s topography, we’re able to take the elevation of a place into account so we can more accurately display the location of the destination pin in Live View. Below, you can see how Lombard Street—a steep, winding street in San Francisco—previously appeared far off into the distance. Now, you can quickly see that Lombard Street is much closer and the pin is aligned with where the street begins at the bottom of the hill.
“If your dog woke up 10 times its current size, it would lick you; if your cat woke up 10 times bigger, it would eat you,” or so I’ve heard.
In any case, building on the world’s viral (in every sense) adoption of AR animals already in search, my team has added a bunch more:
The Verge writes,
When Google started putting 3D animals in Search last year it only had a few standard animals available like a tiger, a lion, a wolf, and a dog. It added more creatures in March, including alligators, ducks, and hedgehogs. In August, Google made prehistoric creatures and historical artifacts available in AR via its Arts and Culture app— and who among us wouldn’t love to check out the ancient crustacean Cambropachycope up close and personal?
Meanwhile my man Seamus abides. 🐕😌
Remember Obama’s first term, when faked tilt-shift photos were so popular that Instagram briefly offered a built-in feature for applying the look? The effect got burned out, but I found it surprisingly fun to see it return in this short video.
In a brief interview, Sofia-based photographer Pavel Petrov shares some behind-the-scenes details.
I have used Adobe Premiere Pro for post processing with some compound blur (for the narrow depth of field) and some oversaturation and speed up to 300%.
Howwwww is this possible, and how does the pilot not eat it immediately and repeatedly?
From sign language to sports training to AR effects, tracking the human body unlocks some amazing possibilities, and my Google Research teammates are delivering great new tools:
We are excited to announce MediaPipe Holistic, […] a new pipeline with optimized pose, face and hand components that each run in real-time, with minimum memory transfer between their inference backends, and added support for interchangeability of the three components, depending on the quality/speed tradeoffs.
When including all three components, MediaPipe Holistic provides a unified topology for a groundbreaking 540+ keypoints (33 pose, 21 per-hand and 468 facial landmarks) and achieves near real-time performance on mobile devices. MediaPipe Holistic is being released as part of MediaPipe and is available on-device for mobile (Android, iOS) and desktop. We are also introducing MediaPipe’s new ready-to-use APIs for research (Python) and web (JavaScript) to ease access to the technology.
Check out the rest of the post for details, and let us know what you create!
7:30pm Pacific time, streaming free via YouTube:
In this talk, we’ll discuss the current state of AI-generated imagery, including Deepfakes and GANs: how they work, their capabilities, and what the future may hold. We’ll try to separate the hype from reality, and examine the social consequences of these technologies with a special focus on the effect that the idea of Deepfakes has had on the public. We’ll consider the visual misinformation landscape more broadly, including so-called “shallowfakes” and “cheapfakes” like Photoshop. Finally, we’ll review the challenges and promise of the global research community that has emerged around detecting visual misinformation.
I don’t know how one encapsulates the paradoxical “glacial/speed-run” feeling of these last 12 months, but the Goog gamely tries in these heartfelt three minutes. For more about the questions that preoccupied the world, check out The Year in Search site.
Researchers at Facebook & universities have devised a way to estimate depth from regular (monocular) video, enabling some beautiful AR effects. Check it out:
A couple of exciting new features have landed for Pixel users. My colleague Navin Sarma writes,
Sky palette transfer in Photos – Sky palette transfer allows users to quickly improve their images that contain sky, achieving a dramatic, creative, and professional effect. It localizes the most dramatic changes to color and contrast to the sky, and tapers the effect to the foreground. It’s especially powerful to improve images of sunsets or sunrises, or where there are complex clouds and contrasty light.
Dynamic/HDR in Photos – The “Dynamic” suggestion is geared towards landscape and “still life” photography, where images can benefit from enhanced brightness, contrast, and color. This effect uses local tone mapping, which allows more control of where brightness and contrast changes occur, making it especially useful in tricky lighting situations. You can use this effect on any photo by using the “Dynamic” suggestion, or navigating to Adjust and moving the “HDR” slider.
Call it AI, ML, FM (F’ing Magic), whatever: tech like this warms the heart and can free body & soul. Google’s Project Guideline helps people with impaired vision navigate the world on their own, independently & at speed. Runner & CEO Thomas Panek, who is blind, writes,
In the fall of 2019, I asked that question to a group of designers and technologists at a Google hackathon. I wasn’t anticipating much more than an interesting conversation, but by the end of the day they’d built a rough demo […].
I’d wear a phone on a waistband, and bone-conducting headphones. The phone’s camera would look for a physical guideline on the ground and send audio signals depending on my position. If I drifted to the left of the line, the sound would get louder and more dissonant in my left ear. If I drifted to the right, the same thing would happen, but in my right ear. Within a few months, we were ready to test it on an indoor oval track. […] It was the first unguided mile I had run in decades.
Check out the journey. (Side note: how great is “Blaze” as a name for a speedy canine running companion? ☺️)
I’ll admit that for some reason I didn’t pay much attention to this new set (the largest-ever by sheer number of bricks) when it emerged a few weeks ago. My history-loving wife took a shine to it, however, and the short vid below shows off an incredible level of detail (those columns tho) and thoughtfulness:
I mean, AR is cool and all, but have we considered a trash bin with some duct tape on it as a helmet? 😂
[Via]
While camping at the funky Sierra Circles sculpture garden/pottery studio/winery, this past weekend, we came across an old Linotype machine hanging out in a field—one of 40+ presses that once existed there, before most were sold to China for scrap. Here’s a tiny gallery I captured:
This reminded my wife to share with me a cool two-minute portrait of America’s last newspaper still printed on the old gear:
…or about the year, at any rate. This is <chef’s kiss>:
Excellent follow-up in a reply:
Researchers at Google, Facebook, and the University of Washington have devised “a fully automatic method for converting a still image into a realistic animated looping video.”
We target scenes with continuous fluid motion, such as flowing water and billowing smoke. Our method relies on the observation that this type of natural motion can be convincingly reproduced from a static Eulerian motion description… We propose a novel video looping technique that flows features both forward and backward in time and then blends the results.
The results are rather amazing.
As I’ve happily noted, this long-gestating feature has rewarded my patience & then some. I’ve found the refinement & control to be terrific.
In the vid below, Aaron Nace shows how to analyze a scene’s lighting, pick a suitable replacement sky, brush to refine edges, tweak the generated masks, and more.
[Via]
There’s no way the title can do this one justice, so just watch as this ML-based technique identifies moving humans (including their reflections!), then segments them out to enable individual manipulation—including syncing up their motions and even removing people wholesale:
Here’s the vid directly from the research team, which includes longtime Adobe vet David Salesin:
