I used to enjoy busting out that humblebrag on Google Photos (and before that, at Adobe) where I got to work with John Schlag. Troubles in the VFX industry yielded a windfall of imaging talent for Google (which occupies the former LA office of Rhythm & Hues, FWIW), and we had a real murderer’s row of talent from DreamWorks, PDI, Sony, and other shops. (There’s so much potential yet to realize… but I digress.)
Are Irish eyes smiling? I should ask my teammate Avneesh to scan me & find out:
Google Research presents a machine learning based approach to infer select facial action units and expressions entirely by analyzing a small part of the face while the user is engaged in a virtual reality experience. Specifically, images of the user’s eyes captured from an infrared (IR) gaze-tracking camera within a VR headset are sufficient to infer at least a subset of facial expressions without the use of any external cameras or additional sensors.
Photographer Jesse Watson captured 2452 images with his wide-angle Nikon D810 setup, culling & cropping the results into a 6K time-lapse using After Effects and Premiere Pro. Check out the beautiful results (and other geeky details here):
You know how Google Assistant can say “Hey, [stateyourname], you should probably leave for the airport by 5 to make it in time for your 7 o’clock flight?” I want it to also say, “You know, it’s Mother’s Day on Sunday. Would you like this photo book to show up on your mom’s doorstep then together with some nice flowers?” Take my money, robot; make me into a better son!
Clearly such work involves a lot of moving parts & hard-to-define qualities (e.g. whether the memories evoked by an image are happy or sad may change greatly depending on things entirely outside the pixels). On the visual quality front, however, my teammates are making interesting progress. As Engadget writes,
If Google has its way, AI may serve as an art critic. It just detailed work on a Neural Image Assessment (NIMA) system that uses a deep convolutional neural network to rate photos based on what it believes you’d like, both technically and aesthetically. It trains on a set of images based on a histogram of ratings (such as from photo contests) that give a sense of the overall quality of a picture in different areas, not just a mean score or a simple high/low rating.
Dwayne Collins’ daughter was diagnosed with a rare eye condition, and none of the doctors they saw could help in any meaningful way. He turned to training videos on YouTube to learn techniques for building prosthetic eyes. It worked—and now he owns his own prosthetics clinic.
In 2017, the world asked “how.” Questions like how to join the military, how to run for office, how to make a protest sign, how to be a good parent, and how to be a firefighter were asked more than ever before. Explore the moments that shaped the year.
Wait—I work on augmented reality tech at Google, so why am I mentioning tools from the other guys?
Well, I’m fundamentally interested in any tools that give people creative superpowers. Just as features like semantic segmentation promise to drain the drudgery from traditional imaging work, tools like these promise to deepen the canvas, letting regular people create experiences that only a few years ago were barely imagined. Good times!
“Build augmented reality experiences for the Facebook camera and Live broadcasts with AR Studio, our customizable suite of creative tools for Mac.”
“Create stunning effects that respond to motion, gestures, facial expressions, and your surroundings. Develop creative effects that instantly respond to comments and reactions during Live broadcasts. Combine data and design to create new immersive experiences for the audiences that matter to you.”
“Everything you need to build and launch immersive Lenses — all in one place.”
“Create and publish your Lens in three easy steps: Design your 3D creation in your favorite software, then import the file right into Lens Studio. Bring your creation to life with movement patterns, animations, and interactive triggers! Preview your Lens on your mobile device, then submit it for publication on Snapchat. If approved, you’ll get a unique Snapcode anyone can scan to unlock your Lens. Then, your Lens can be sent directly to friends, and unlocked again!”
An immense, whooshing kinetic sculpture darts over the Netherlands, brought to you by thousands of beating hearts & flapping wings:
The art of flying is a short film about “murmurations”: the mysterious flights of the Common Starling. It is still unknown how the thousands of birds are able to fly in such dense swarms without colliding. Every night the starlings gather at dusk to perform their stunning air show.
Because of the relatively warm winter of 2014/2015, the starlings stayed in the Netherlands instead of migrating southwards. This gave filmmaker Jan van IJken the opportunity to film one of the most spectacular and amazing natural phenomena on earth.
I’m excited for my teammates & their new launches. The team writes,
Storyboard (Android) transforms your videos into single-page comic layouts, entirely on device. Simply shoot a video and load it in Storyboard. The app automatically selects interesting video frames, lays them out, and applies one of six visual styles.
Selfissimo! (iOS, Android) is an automated selfie photographer that snaps a stylish black and white photo each time you pose. Tap the screen to start a photoshoot. The app encourages you to pose and captures a photo whenever you stop moving.
Scrubbies (iOS) lets you easily manipulate the speed and direction of video playback to produce delightful video loops… Shoot a video in the app and then remix it by scratching it like a DJ. Scrubbing with one finger plays the video. Scrubbing with two fingers captures the playback so you can save or share it.
Please take ‘em for a spin, then tell us what you think using the in-app feedback links.
The idea behind the film was to find the innate movement inherit in still forms. Every sculpture has movement in it, and it is the task of the animator to discover it. It was through the process of editing my imagery that I discovered that a single image would suffice to create the animation. The film was made by zooming into the image and panning row by row while making sure that different architectural motives aligned in every increment. This also gave a structure to the film.
“Being early is the same as being wrong,” says Marc Andreessen. True enough.
I have to admit that when I saw Photoshop’s “new” 360º photo-editing feature, I was a little miffed at the positioning: we shipped almost the same exact thing nine years ago. PS has now bolted on a few menu items to make access more obvious, but otherwise the tech appears largely unchanged.
I get it, though. Nine years ago, how would one create such an image (tripod, SLR, stitching package?) and where would one use it? Now the ecosystem is radically different: You can capture an image in an instant via a Theta or similar cam (or even a drone!), or you can capture one with any smartphone, and you can make them interactively explorable by millions of people via Facebook. So yeah, viewed through that lens, I get it, and I hope that orders of magnitude more people find this feature useful this time around. ⚪️💪
Speaking of tech ahead of its time (?), I hope that wearable capture devices will become practical & enable the kind of experience that Googlers Blaise Agüera y Arcas & Noah Snavely pioneered at Microsoft:
I find this Quixotic quest to be roughly 50 kinds of charming:
When Tatsuo Horiuchi retired, he decided to try his hand at art. But instead of spending money on paints and brushes, Horiuchi used what he already had pre-installed on his computer—Microsoft Excel. Now, the 77-year-old artist is creating remarkably intricate digital masterpieces of the Japanese landscape, all on the free graphing software.
How might the world look if populated by 3D shapes tied to musical beats? Artist Oscar López Rocha imagines it:
I followed the “karaoke” principle in this project, lyrics appearing while music is playing. Instead of these, I’ve changed them for geometrical bodies appearing to the rhythm of a song to create visual 3D compositions in real places where I’ve passed by sometime and captured in DSLR video.
When a new movie comes out, most of the praise goes to the director and the lead actors, but there are so many other people involved in a film, and a lot of them are designers. There are costume designers and set designers, but also graphic designers working behind the scenes on every single graphic object that you might need in a film. It’s Annie Atkins’s job to design them.