Monthly Archives: February 2018

Take a 360º journey through the “eyes” of a self-driving Waymo car

Google’s self-driving car sibling has the world’s only fleet of fully self-driving cars on public roads. They built this short film using footage and real-time data from an actual trip on city streets:

Step into our 360° video and take control of the camera to see through the “eyes” of our car. Then, be one of the first in the world to take a ride with Waymo.

Here’s how to explore the video in 360°:

  • On mobile, move your phone around to explore in all directions. 
  • On desktop, use your mouse to drag the video around your screen. 
  • If you have Cardboard, tap the Cardboard icon in the bottom right of the YouTube player, insert your phone into Cardboard, and look around. If you have Google Daydream View, just place your phone into the headset.




Skydio AI drone promises “a flim crew that fits in a backpack”

Skydio boasts 13 cameras that power really impressive collision-avoidance tech, allowing it to track a person even through obstacles like woods. Check it out:

Here’s how the “Autonomy Engine” works:

The device looks bulky, but it’s said to fit into any backpack that can accommodate a 17-inch laptop. Being roughly 3x the price ($2,499), size, and weight of a Mavic Air, the Skydio makes me ask a few questions:

  • What’s the average utilization of any drones people buy? They seem like action-sports cameras: aspirational, highly specialized, rarely used. (Thus it made perfect sense to me that GoPro would get into the drone business, and that doing so might just compound their existing problems.)
  • How important is this kind of aerial selfie mode that really sets Skydio apart? That is, what percentage of the time one wants to use a drone is it for, say, mountain biking through the woods? The obvious concern is that it falls into a real niche (the small Venn diagram overlap of “actually doing action sports” and “wanting to view from the air”).
Having said all that, the AI capabilities look like a great step forward, and I’m eager to learn more as the device starts reaching customers.


[YouTube 1 and 2]

Explore England’s Enormous Chalk Figures

Google Earth has teamed up with Atlas Obscura to provide an interactive tour of England’s giant chalk figures—from massive horses to, um, excited giants. Check it out here. Funky details abound:

From the 17th to the 19th centuries, routine maintenance of the horse took the form of “scouring” festivals every seven years, which included not only repairs but other entertaining activities such as cheese-wheel chasing and pipe-smoking marathons.


In 2007, an enormous Homer Simpson carving, complete with a very large doughnut, appeared near the Cerne Abbas Giant as a promotion for The Simpsons Movie.


A little drone fun at 10,000 ft.


We’ve just returned from spending a week in Leadville, CO, which at 10,200 ft. is the highest incorporated city in the US. Despite the inevitable dehydration insomnia that always afflicts me there, I didn’t find the elevation quite high enough, so I brought my Mavic Pro (piloted by a custom-printed “Googler” minifig) to capture a few shots of Leadville from above. You can find some 360º panoramas (uploaded to & embedded via Google Maps) and videos below.

Quick observations in case you’d find them useful:


  • DJI Goggles do make it easier to fly while panning/tilting one’s camera, and I got my smoothest shots yet. On the downside, I’ve yet to figure out how to adjust the picture to look fully clear to my eyes (with or without glasses), and the fact that you apparently can’t just put them into head-tracking mode before a flight & keep them there is a pain.
  • 360º panorama capture is totally my jam. Whereas I find capturing video stressful (is it smooth enough? will anyone find it interesting?) and editing video laborious, shooting panos is almost trivially easy. Having said that, I ran into a few snags:
    • Downloading & installing the needed firmware update took several tries, due partly to dodgy mountain WiFi & partly to the DJI app’s less-than-straightforward flow.
    • An inscrutable, non-dismissible (!) warning screen would sometimes pop up to let us know that we were a few miles from an airport. It blocking most of the screen, coupled with the wonky path one must follow to invoke pano capture mode, meant that we blew it on at least one pano, accidentally capturing 180º instead of 360º—and facing the wrong way at that!
    • I found that Facebook wouldn’t recognize the panos as panos, so after some searching I found & downloaded the metadata-tweaking tool Exif Fixer. Running it on the images did the trick.
    • Stitching images on my iPhone X was fairly quick (taking perhaps a minute for a full 360) and did a pretty good job (PTGui, which I downloaded, seemed to do no better). On the downside it sucks that the process is modal, blocking you from using the app to fly, so you’d probably do well to wait & stitch after landing.
    • The exposure on what might’ve been my best pano was totally blown out for reasons I don’t understand. ¯\_(ツ)_/¯
 Okay, on to the pics & vids!

Ballet Rotoscope

This short film from Masahiko Sato + EUPHRATES is poetry in motion:

[B]y extracting the movement of a dancing ballerina, we created a animation of locus drawn in the air, and dancing geometric figures, composed with a documentary picture. The geometric figures were created by connecting the movement of ballerina’s joints by using algorithm of computer geometry(such as “convex hull”,and “Delaunay diagram”). Our aim was to represent a complete new type of beauty, by showing the interaction of abstract animation with realistic movements and documentary film.



Illustration: A charming street-art ad from Coke

I’ve been getting a kick out of this spot running during the Olympics:

The minute-long spot features three animated graffiti characters stealthily shifting off a wall and, unseen by passersby distracted by drinking Coca-Cola, climb their way up skyscrapers towards their destination: a print ad of three bottles of Coca-Cola. The trio then enjoy the drinks together on the side of a water tower.



Mesmerizing drone shots from inside a fireworks cloud

Happy Chinese New Year from above Yunnan, China:

The intriguing reversal of footage reminds me of this beautifully sad passage from Slaughterhouse Five:

The formation flew backwards over a German city that was in flames. The bombers opened their bomb bay doors, exerted a miraculous magnetism which shrunk the fires, gathered them into cylindrical steel containers , and lifted the containers into the bellies of the planes. The containers were stored neatly in racks. The Germans below had miraculous devices of their own, which were long steel tubes. They used them to suck more fragments from the crewmen and planes. But there were still a few wounded Americans though and some of the bombers were in bad repair. Over France though, German fighters came up again, made everything and everybody as good as new.

When the bombers got back to their base, the steel cylinders were taken from the racks and shipped back to the United States of America, where factories were operating night and day, dismantling the cylinders, separating the dangerous contents into minerals. Touchingly, it was mainly women who did this work. The minerals were then shipped to specialists in remote areas. It was their business to put them into the ground, to hide them cleverly, so they would never hurt anybody ever again.


Google releases data to help improve HDR imaging

The smart folks responsible for HDR+ on the Pixel 2 are sharing a big dataset in order to help other developers create better high-dynamic-range imagery:

Today we’re pleased to announce the public release of an archive of image bursts to the research community. This provides a way for others to compare their methods to the results of Google’s HDR+ software running on the same input images. This dataset consists of 3,640 bursts of full-resolution raw images, made up of 28,461 individual images, along with HDR+ intermediate and final results for comparison.

[O]ur hope is that a shared dataset will enable the community to concentrate on comparing results. This approach is intrinsically more efficient than expecting researchers to configure and run competing techniques themselves, or to implement them from scratch if the code is proprietary. 


Rather fascinating: “What ‘Super Mario’ Looks Like at 380,000fps”

This is my kind of nerdery:

Old TVs render an image by actually drawing the entire frame from top to bottom at speeds so fast the human eye can’t detect it, but Free and Gruchy’s cameras can. Mario is almost indistinguishable at 380,000 FPS, but it’s amazing to watch beams of light shoot across a screen in slow motion, slowly building the Mushroom Kingdom we all know and love.


[YouTube] [Via]

Audi’s AR app lets you draw a racetrack in your living room

This app (sadly unavailable in the US, it seems) looks really creative & fun:

“To achieve a seamless transition from the TV ad to Augmented Reality we use computer vision to detect the quattro coaster TV ad. Then, we sync and position the augmented content on the screen. What’s interesting is that the car remains in the room even after the ad has ended. [more]

Here’s what it looks like in action:



[YouTube] [Vimeo]

Motion Stills gains a fun new AR mode on Android

Hooray! My first real project to ship since joining my new team is here:

Today, we are excited to announce the new Augmented Reality (AR) mode in Motion Stills for Android. With the new AR mode, a user simply touches the viewfinder to place fun, virtual 3D objects on static or moving horizontal surfaces (e.g. tables, floors, or hands), allowing them to seamlessly interact with a dynamic real-world environment. You can also record and share the clips as GIFs and videos. 


For the nerdier among us, we’ve put together a Research blog post about The Instant Motion Tracking Behind Motion Stills AR, and on CNET Stephen Shankland gives a nice overview (and has been tweeting out some fun animations):

The Motion Stills app can put AR stickers into any Android device with a gyroscope, which is nothing special these days.

I’ve long been a longtime fan of Motion Stills, posting about it for years. I’m so glad to get to work with these guys now. There’s more good stuff to come, so please us know what you think!

(BTW, the 3D models are among the many thousands you can download for free from

Endless Instagram repetition, the musical!

According to a recent survey, more than 40% of people under 33 prioritize “Instagrammability” when choosing their next holiday spot. Of course, a ton of the results look incredibly similar, perhaps inducing cases of vemödalen (“the frustration of photographing something amazing when thousands of identical photos already exist”). They’re so repetitive, in fact, that Google researchers have built 3D timelapses from overlapping imagery.

Maybe more people need a “Camera Restricta” to “prevent shooting unoriginal photos.” Or maybe we shouldn’t sweat it, because “there are many like it, but this one is mine,” and because “in the end we shall all be dead.” ¯\_(ツ)_/¯



Google visualizes thousands of bird sounds via AI

This thing is pretty cool! Cornell researchers worked with Googlers to use machine learning in order to fingerprint the songs of various birds, then lay them out in an interactive visualization:



Built by Kyle McDonald, Manny Tan, Yotam Mann, and friends at Google Creative Lab. Thanks to Cornell Lab of Ornithology for their support. The Essential Set for North America sounds are provided by the Macaulay Library. The open-source code is available here. Check out more at A.I. Experiments.