Category Archives: VR/AR

BumpTop is back… now in AR!

I wish I could find the joke video someone made when Google acquired the beloved email client Sparrow back in 2012. The vid presented itself as a cheerful tutorial on “How to prepare for the Sparrow Google migration” that basically went like this: 

  1. Open your Applications folder.
  2. Locate the Sparrow app icon.
  3. Now, carefully just drag it to the trash.
  4. Empty the trash.
  5. And voila, now you’re ready for Google’s ownership of this great app!

I’ve thought of it countless times over the years, and I think of it now recalling BumpTop, a 3D desktop GUI that I first wrote about 13 years ago (!):

Google acquired the tech & team back in 2010, and then… nothing, as far as I know. (All this predated my time at the company, so I have zero inside info.) Empty the trash, and voila!

But evidently not content to let things die, the BumpTop team has reemerged with Spatial, a new approach to team collaboration done via virtual & augmented reality. Check it out:

Will Spatial have more lasting impact than BumpTop? I have no idea! But I look forward to trying it out, and I’m sure glad that teams like this are busy trying to make the world more delightful.


[YouTube 1 & 2]

Pixel 3 Playground is here!

I’m so pleased to reveal what we’ve been working on for quite some time—the new Playground augmented reality mode on Google Pixel devices!

Playground brings you more powerful AR experiences and uses AI to recommend content for expressing yourself in the moment. You can make your photos and videos come to life with Playmoji—characters that react to each other and to you—and tell a richer story by adding fun captions or animated stickers.

Playground makes real-time suggestions to recommend content based on the scene you’re in. Are you walking your dog? Cooking in the kitchen? Gardening in the backyard? Playground uses advanced computer vision and machine learning to recommend relevant Playmoji, stickers and captions to populate the scene.

My team contributed tech that enables selfie stickers (using realtime segmentation to let characters stand behind you), reactive stickers (those that respond to humans in the frame), object tracking (so that you can attach stickers to moving elements like pets & hands), and glue that helps the pieces communicate. Happily, too, we’re just getting warmed up.

Oh, and stay tuned for Gambeezy!



VR creators: Apply for Google’s Creator Lab in London

Sounds pretty cool:

Today we’re announcing that VR Creator Lab is coming to London. Participants will receive between $30,000 and $40,000 USD in funding towards their VR project, attend a three day “boot camp” September 18-20, 2018, and receive three months of training from leading VR instructors and filmmakers.

Applications are open through 5pm British Summer Time on August 6, 2018. YouTube creators with a minimum of 10,000 subscribers and independent filmmakers are eligible.



AR soccer on your table? Google & Facebook researchers make it happen

Can low-res YouTube footage be used to generate a 3D model of a ballgame—one that can then be visualized from different angles & mixed into the environment in front of you? Kinda, yeah!

Per TechCrunch,

The “Soccer On Your Tabletop” system takes as its input a video of a match and watches it carefully, tracking each player and their movements individually. The images of the players are then mapped onto 3D models “extracted from soccer video games,” and placed on a 3D representation of the field. Basically they cross FIFA 18 with real life and produce a sort of miniature hybrid.



VR180 Creator, a new video tool from Google

Sounds handy for storytellers embracing new perspectives:

VR180 Creator currently offers two features for VR videos. “Convert for Publishing” takes raw fisheye footage from VR180 cameras like the Lenovo Mirage Camera and converts it into a standardized equirect projection. This can be edited with the video editing software creators already use, like Adobe Premiere and Final Cut Pro. “Prepare for Publishing” re-injects the VR180 metadata after editing so that the footage is viewable on YouTube or Google Photos in 2D or VR.

You can learn more about how to use VR180 Creator here and you can download it here.



“Bumping the Lamp”: AR storytelling insights from Roger Rabbit (for real!)

If you’re interested in making augmented reality characters feel natural in the real world, it’s well worth spending a few minutes with this tour of some key insights. I’ve heard once-skeptical Google AR artists praising it, saying, “This video is a treasure trove and every artist, designer or anyone working on front-end AR should watch it.” Enjoy, and remember to bump that lamp. 🙂 


[YouTube] [Via Jeremy Cowles]