Category Archives: VR/AR

Make a 3D model of your face just by uploading a photo

Creeptastic! But quick, cool, and impressive: Visit The University of Nottingham’s demo site, and check out the project site for more details. As The Verge writes,

“3D face reconstruction is a fundamental computer vision problem of extraordinary difficulty.” You usually need multiple pictures of the same face from different angles in order to map every contour. But, by feeding a bunch of photographs and corresponding 3D models into a neural network, the researchers were able to teach an AI system how to quickly extrapolate the shape of a face from a single photo.

NewImage

[Via Alex Kauffmann]

Google Earth VR adds Street View

Looks awesome:

This update lets you explore Street View imagery from 85 countries right within Earth VR. Just fly down closer to street level, check your controller to see if Street View is available and enter an immersive 360° photo. You’ll find photos from the Street View team and those shared by people all around the world.

NewImage

DubrovGif

“Inside Music”: A WebVR sonic explorer from Google & Sound Exploder

Check out this multi-track song explorer from a cool podcast (see previous) & Google’s WebVR team:

What if you could step inside a song? Inside Music is a simple experiment that explores that idea. It features the music of Phoenix, Natalia Lafourcade, Perfume Genius, Alarm Will Sound, Clipping, and Ibeyi. 

if you’re a musician, you can explore your own songs in VR or put them up on the web for others to explore.

NewImage

[YouTube 1 and 2] [Via]

Google introduces ARCore, Web augmented reality previews

I’m delighted that Google releasing a preview SDK of ARCore, bringing augmented reality capabilities to existing and future Android phones. Developers can start experimenting with it right now.

The team writes,

It works without any additional hardware, which means it can scale across the Android ecosystem. ARCore will run on millions of devices, starting today with the Pixel and Samsung’s S8, running 7.0 Nougat and above. We’re targeting 100 million devices at the end of the preview.

And:

We’re also working on Visual Positioning Service (VPS), a service which will enable world scale AR experiences well beyond a tabletop. And we think the Web will be a critical component of the future of AR, so we’re also releasing prototype browsers for web developers so they can start experimenting with AR, too. These custom browsers allow developers to create AR-enhanced websites and run them on both Android/ARCore and iOS/ARKit.

Super exciting times!

[YouTube]

Disney Magic Bench: AR you can feel

The miracles & wonder continue:

Our approach is to instrument the environment leaving the user unburdened of any equipment, creating a seamless walk-up-and-play experience. We demonstrate this technology in a series of vignettes featuring humanoid animals.

Participants can not only see and hear these characters, they can also feel them on the bench through haptic feedback. Many of the characters also interact with users directly, either through speech or touch.

NewImage

NewImage

[YouTube]

Google & Major League Baseball team up on VR

When Periscope debuted two years ago, I thought it would quickly usher in an era of live multi-camera feeds of clubhouse champagne parties, dugout conversations, and more. Maybe it still will, but for most uses, it became clear (as maybe it should have been at the start) that very little content demands or rewards live viewing. The Snapchat-pioneered stories format, by contrast, feels live but offers just enough curation & control for both creators & viewers that it’s dramatically more compelling for most occasions.

Anyway, the jury remains out on VR, but I’m glad to see more experiments getting underway. Now Google is using the compact Jump camera rig to partner with MLB in telling young players’ stories. Engadget writes,

[T]he NFL partnered with Google Daydream to produce an exclusive series called All or Nothing last year. Major League Baseball has also collaborated with Google Daydream on a video game and the MLB.com At Bat app. Now, they’ve announced their latest partnership: “On the Verge,” which is a VR series that profiles up-and-coming baseball stars.

The first episodes are available now. They’ll also be available on MLB’s official YouTube account soon.

NewImage

NewImage

YouTube VR: 360º heatmaps, Creators Lab

Viewer: “Where the hell should I look?”
Creator: “Where the hell do people look?”

Making compelling 360º content—like both pimpin’ & impin’—ain’t easy. Fortunately YouTube is adding some new analytical tools:

Today we’re introducing heatmaps for 360-degree and VR videos with over 1,000 views, which will give you specific insight into how your viewers are engaging with your content. With heatmaps, you’ll be able to see exactly what parts of your video are catching a viewer’s attention and how long they’re looking at a specific part of the video.

 Meanwhile they’ve started a new VR Creator Lab bootcamp:

Take your VR video creation to the next level. YouTube is taking applications for a 3 month learning and production intensive for VR creators. Participants will receive advanced education from leading VR instructors, 1:1 mentoring, and $30K – $40K in funding toward the production of their dream projects.

The application window has now closed (sorry I didn’t the news ’til now), but hopefully this will go well & future openings will emerge.

NewImage

Lightform promises to bring AR into physical spaces

The makers of Lightform call it “the first computer made for projected augmented reality.”

Lightform scans complex scenes in under a minute, letting you seamlessly mix real objects with projected light. It’s augmented reality without the headset.

Check out a demo made with it & read more on Wired:

The small box contains a processor and a high-res camera. Hook it up to any projector through an HDMI cable, and the projector will cast a series of grids onto the room, which Lightform’s onboard camera uses to assess, in fine detail, the location and dimensions of objects in the space. (Lightform can also scan the room periodically, allowing it to create a new map if anything moves.) The processor converts that information into a 3-D map of surfaces onto which the projector can cast light. […]

In other words: Lightform helps you quickly transform almost anything in a room into a screen

NewImage

Google introduces Blocks, a VR model creation app

Check out Blocks for Vive & Oculus Rift:

You can browse example content and read Fast Company’s coverage, “Google Is Becoming The Adobe Of VR”:

Google is essentially modeling Adobe to fill some of Adobe’s own gaps. First, it acquired Tiltbrush for VR sketching. Now, it built Blocks for VR-based, 3D object creation…

Google is laying the foundation for a massive play in VR and AR, because Blocks will be the cornerstone of an Adobe-like suite of VR creation apps from Google, which will pave the way for a new wave of user-created 3D movies and interactive experiences to come.

NewImage

NewImage

[YouTube]