Category Archives: VR/AR

Google releases open-source, browser-based BodyPix 2.0

Over the last couple of years I’ve pointed out a number of cool projects (e.g. driving image search via your body movements) powered by my teammates’ efforts to 1) deliver great machine learning models, and 2) enable Web browsers to run them efficiently. Now they’ve released BodyPix 2.0 (see live demo):

We are excited to announce the release of BodyPix, an open-source machine learning model which allows for person and body-part segmentation in the browser with TensorFlow.js. With default settings, it estimates and renders person and body-part segmentation at 25 fps on a 2018 15-inch MacBook Pro, and 21 fps on an iPhone X.

Enjoy!

“Extreme Augmented Reality”: Nvidia connects cloud & handset

Instead of squeezing models down to a couple of megs & constraining rendering to what a phone alone can do, what if you could use a full-power game engine to render gigabyte-sized models in realtime in the cloud, streaming the results onto your device to combine with the world? That’s the promise of Nvidia CloudXR (which looks similar to Microsoft Azure Remote Rendering), announced this week:

[YouTube]

Become Maleficent with some YouTube AR

Check out this video on your compatible iPhone or Android device to don the horns & makeup of this shady lady, powered by the augmented reality tech my team has been building.

NewImage

[YouTube]

The Micronaxx go AR

Much like his big bro did last year (see below), our son Henry stepped up to el micrófono to tell tales of meteorological mayhem. It took a village, with mom scoring a green-screen kit from her Adobe video pals & me applying some AR effects created by my talented teammates.

Here’s a behind-the-scenes peek at our advanced VFX dojo/living room. 😌

NewImage