Category Archives: VR/AR

The Micronaxx go AR

Much like his big bro did last year (see below), our son Henry stepped up to el micrófono to tell tales of meteorological mayhem. It took a village, with mom scoring a green-screen kit from her Adobe video pals & me applying some AR effects created by my talented teammates.

Here’s a behind-the-scenes peek at our advanced VFX dojo/living room. 😌


Google creates a dataset to help detect deepfakes

“With great power…” I’m pleased to see some of my collaborators in augmented reality working to help fight deceptive content:

To make this dataset, over the past year we worked with paid and consenting actors to record hundreds of videos. Using publicly available deepfake generation methods, we then created thousands of deepfakes from these videos. The resulting videos, real and fake, comprise our contribution, which we created to directly support deepfake detection efforts. As part of the FaceForensics benchmark, this dataset is now available, free to the research community, for use in developing synthetic video detection methods.

Snapchat introduces 3D selfies

Looks fun, though I have no idea how to create these; “open the app to the camera, navigate to the 3D option option in the dropdown menu, and voila:

The Verge writes,

Starting today, people with an iPhone X or newer can use “3D Camera Mode” to capture a selfie and and apply 3D effects, lenses, and filters to it.

Snap first introduced the idea for 3D effects with Snaps when it announced its latest version of Spectacles, which include a second camera to capture depth. The effects and filters add things like confetti, light streaks, and miscellaneous animations.


Use Google’s AR face tech to build cross-platform effects

Look Ma, no depth sensor required.

People seem endlessly surprised that one is not only allowed to use an iPhone at Google, but that we also build great cross-platform tech for developers (e.g. ML Kit). In that vein I’m delighted to say that my team has now released an iOS version (supporting iPhone 6s and above) of the Augmented Faces tech we first released for ARCore for Android earlier this year:

It provides a high-quality, 468-point 3D mesh that lets users attach fun effects to their faces — all without a depth sensor on their smartphone. With the addition of iOS support rolling out today, developers can now create effects for more than a billion users. We’ve also made the creation process easier for both iOS and Android developers with a new face effects template.

Here’s a quick overview from my teammate Sam:


You can now Lens Uncle Ben’s to get personalized info

I know this will seem like small beans—literally—but over time it’ll be a big deal, and not just because it’s an instance of the engine I’m working to enhance.

Through Lens, you’ll get meal recommendations based on your tastes, dietary preferences, and allergies, along with a personalized score for products like Uncle Ben’s Ready Rice, Flavored Grains, Flavor Infusions, and beans.

VentureBeat goes on to note,

The growing list of things Lens can recognize covers over 1 billion products… The new feature follows a Lens capability that highlights top meals at a restaurant and a partnership with Wescover that supplies information about art and design installations. Lens also recently gained the ability to split a bill or calculate a tip after a meal; [and] to overlay videos atop real-world publications.

Check out the latter, from a couple of months ago. As I say, big things have small beginnings.