Wherever you are & whatever you may celebrate, I hope you and yours are well this Christmas. I’ve been amusing the fam by dialing up some old Chieftains Christmas tracks (which transport me back to my relatives’ cozy homes in Chicago growing up), including this absolute banger. 😌 Enjoy.
Although we can’t travel far this holiday season (bye bye, Mendocino! see you… someday?), we are, in the words of Tony Stark, “bringing the party to you” via our decked-out VW Westy (pics). I’m having fun experimenting with my new Insta360 One X2, mounting it on a selfie stick & playing with timelapse mode. Here’s a taste of the psychedelic stylings, courtesy of The Weeknd…
Apropos of nothing, check out 60 lovingly rendered seconds commissioned by YouTube:
Maciej Kuciara writes,
MECHA – the love letter to our youth. Watching anime classics as kids left a mark that stayed with us to this day. So we felt it’s due to time to celebrate our love to mecha and pay proper homage with this piece we did for YouTube.
During the course of 400 hours, Mitsui snapped together 50,000 cobalt and white LEGO into an undulating wave that mimics the original woodblock print.
To recreate this iconic work in three-dimensions, Mitsui studied videos of waves crashing and pored over academic papers on the topic. He then sketched a detailed model before assembling the textured water, three boats, and Mount Fuji that span more than five feet.
I’m delighted to be closing out 2020 on a pair of high notes, welcoming the arrival of my two biggest efforts from the last year+.
First, Google Search now supports 150+ new cars that you can view in 3D and AR (via iPhone or Android device), including in beautiful cloud-rendered quality (provided you have a good connection & up-to-date Android). As we initially previewed in October:
Bring the showroom to you with AR
You can easily check out what the car looks like in different colors, zoom in to see intricate details like buttons on the dashboard, view it against beautiful backdrops and even see it in your driveway. We’re experimenting with this feature in the U.S. and working with top auto brands, such as Volvo and Porsche, to bring these experiences to you soon.
Now, when you search for a lipstick or eyeshadow product, like L’Oreal’s Infallible Paints Metallic Eyeshadow, you can see what it looks like on a range of skin tones and compare shades and textures to help you find the right products.
To help you find the perfect match, you can now also virtually try makeup products right from the Google app.
Google researchers Ira Kemelmacher-Shlizerman, Brian Curless, and Steve Seitz have been working with University of Washington folks on tech that promises “30fps in 4K resolution, and 60fps for HD on a modern GPU.”
Our technique is based on background matting, where an additional frame of the background is captured and used in recovering the alpha matte and the foreground layer.
Nearly 20 years ago, on one of my first customer visits as a Photoshop PM, I got to watch artists use PS + After Effects to extract people from photo backgrounds, then animate the results. The resulting film—The Kid Stays In The Picture—lent its name to the distinctive effect (see previous).
Now I’m delighted that Google Photos is rolling out similar output to its billion+ users, without requiring any effort or tools:
We use machine learning to predict an image’s depth and produce a 3D representation of the scene—even if the original image doesn’t include depth information from the camera. Then we animate a virtual camera for a smooth panning effect—just like out of the movies.
Photos is also rolling out new collages, like this:
And they’re introducing new themes in the stories-style Memories section up top as well:
Now you’ll see Memories surface photos of the most important people in your life… And starting soon, you’ll also see Memories about your favorite things—like sunsets—and activities—like baking or hiking—based on the photos you upload.
I love these simple, practical uses of augmented reality. The Maps team writes,
Last month, we launched Live View in Location Sharing for Pixel users, and we’ll soon expand this to all Android and iOS users around the globe. When a friend has chosen to share their location with you, you can easily tap on their icon and then on Live View to see where and how far away they are–with overlaid arrows and directions that help you know where to go.
Live View in Location Sharing will soon expand to all Android and iOS users globally on ARCore and ARKit supported phones.
They’re also working hard to leverage visual data & provide better localization and annotation.
With the help of machine learning and our understanding of the world’s topography, we’re able to take the elevation of a place into account so we can more accurately display the location of the destination pin in Live View. Below, you can see how Lombard Street—a steep, winding street in San Francisco—previously appeared far off into the distance. Now, you can quickly see that Lombard Street is much closer and the pin is aligned with where the street begins at the bottom of the hill.
When Google started putting 3D animals in Search last year it only had a few standard animals available like a tiger, a lion, a wolf, and a dog. It added more creatures in March, including alligators, ducks, and hedgehogs. In August, Google made prehistoric creatures and historical artifacts available in AR via its Arts and Culture app— and who among us wouldn’t love to check out the ancient crustacean Cambropachycope up close and personal?
Remember Obama’s first term, when faked tilt-shift photos were so popular that Instagram briefly offered a built-in feature for applying the look? The effect got burned out, but I found it surprisingly fun to see it return in this short video.
In a brief interview, Sofia-based photographer Pavel Petrov shares some behind-the-scenes details.
I have used Adobe Premiere Pro for post processing with some compound blur (for the narrow depth of field) and some oversaturation and speed up to 300%.
From sign language to sports training to AR effects, tracking the human body unlocks some amazing possibilities, and my Google Research teammates are delivering great new tools:
We are excited to announce MediaPipe Holistic, […] a new pipeline with optimized pose, face and hand components that each run in real-time, with minimum memory transfer between their inference backends, and added support for interchangeability of the three components, depending on the quality/speed tradeoffs.
In this talk, we’ll discuss the current state of AI-generated imagery, including Deepfakes and GANs: how they work, their capabilities, and what the future may hold. We’ll try to separate the hype from reality, and examine the social consequences of these technologies with a special focus on the effect that the idea of Deepfakes has had on the public. We’ll consider the visual misinformation landscape more broadly, including so-called “shallowfakes” and “cheapfakes” like Photoshop. Finally, we’ll review the challenges and promise of the global research community that has emerged around detecting visual misinformation.
I don’t know how one encapsulates the paradoxical “glacial/speed-run” feeling of these last 12 months, but the Goog gamely tries in these heartfelt three minutes. For more about the questions that preoccupied the world, check out The Year in Search site.
A couple of exciting new features have landed for Pixel users. My colleague Navin Sarma writes,
Sky palette transfer in Photos – Sky palette transfer allows users to quickly improve their images that contain sky, achieving a dramatic, creative, and professional effect. It localizes the most dramatic changes to color and contrast to the sky, and tapers the effect to the foreground. It’s especially powerful to improve images of sunsets or sunrises, or where there are complex clouds and contrasty light.
Dynamic/HDR in Photos – The “Dynamic” suggestion is geared towards landscape and “still life” photography, where images can benefit from enhanced brightness, contrast, and color. This effect uses local tone mapping, which allows more control of where brightness and contrast changes occur, making it especially useful in tricky lighting situations. You can use this effect on any photo by using the “Dynamic” suggestion, or navigating to Adjust and moving the “HDR” slider.
Call it AI, ML, FM (F’ing Magic), whatever: tech like this warms the heart and can free body & soul. Google’s Project Guideline helps people with impaired vision navigate the world on their own, independently & at speed. Runner & CEO Thomas Panek, who is blind, writes,
In the fall of 2019, I asked that question to a group of designers and technologists at a Google hackathon. I wasn’t anticipating much more than an interesting conversation, but by the end of the day they’d built a rough demo […].
I’d wear a phone on a waistband, and bone-conducting headphones. The phone’s camera would look for a physical guideline on the ground and send audio signals depending on my position. If I drifted to the left of the line, the sound would get louder and more dissonant in my left ear. If I drifted to the right, the same thing would happen, but in my right ear. Within a few months, we were ready to test it on an indoor oval track. […] It was the first unguided mile I had run in decades.
Check out the journey. (Side note: how great is “Blaze” as a name for a speedy canine running companion? ☺️)
I’ll admit that for some reason I didn’t pay much attention to this new set (the largest-ever by sheer number of bricks) when it emerged a few weeks ago. My history-loving wife took a shine to it, however, and the short vid below shows off an incredible level of detail (those columns tho) and thoughtfulness:
While camping at the funky Sierra Circles sculpture garden/pottery studio/winery, this past weekend, we came across an old Linotype machine hanging out in a field—one of 40+ presses that once existed there, before most were sold to China for scrap. Here’s a tiny gallery I captured:
Researchers at Google, Facebook, and the University of Washington have devised “a fully automatic method for converting a still image into a realistic animated looping video.”
We target scenes with continuous fluid motion, such as flowing water and billowing smoke. Our method relies on the observation that this type of natural motion can be convincingly reproduced from a static Eulerian motion description… We propose a novel video looping technique that flows features both forward and backward in time and then blends the results.
There’s no way the title can do this one justice, so just watch as this ML-based technique identifies moving humans (including their reflections!), then segments them out to enable individual manipulation—including syncing up their motions and even removing people wholesale:
Here’s the vid directly from the research team, which includes longtime Adobe vet David Salesin: