Category Archives: AR/VR

New Adobe tech promises 3D & materials scanning

Probably needless to say, 3D model creation remains hard AF for most people, and as such it’s a huge chokepoint in the adoption of 3D & AR viewing experiences.

Fortunately we may be on the cusp of some breakthroughs. Apple is about to popularize LIDAR on phones, and with it we’ll see interesting photogrammetry apps like Polycam:

Meanwhile Adobe is working to enable 3D scanning using devices without fancy sensors. Check out Project Scantastic:

They’re also working to improve the digitization of materials—something that could facilitate the (presently slow, expensive) digitization of apparel:

Snap puts the AR in graffiti art

The notion of a metaverse, “a collective virtual shared space, created by the convergence of virtually enhanced physical reality and physically persistent virtual space,” has long beguiled those of us captivated by augmented reality. Now Snap has been doing the hard work of making this more real, being able to scan & recognize one’s surroundings and impose a “persistent, shared AR world built right on top of your neighborhood.” Check it out:

This experience (presently available on just one street in London, but presumably destined to reach many others) builds on the AR Landmarkers work the company did previously. (As it happens, I think David Salesin—who led Adobe Research for many years—contributed to this effort during his stopover at Snap before joining Google Research.)

Come see the AR feature I’ve been working on all year!

I’m delighted to share that my team’s work to add 3D & AR automotive results to Google Search—streaming in cinematic quality via cloud rendering—has now been announced! Check out the demo starting around 36:30:

Here’s how we put it on via the Google Keyword blog:

Bring the showroom to you with AR

You can easily check out what the car looks like in different colors, zoom in to see intricate details like buttons on the dashboard, view it against beautiful backdrops and even see it in your driveway. We’re experimenting with this feature in the U.S. and working with top auto brands, such as Volvo and Porsche, to bring these experiences to you soon.

Cloud streaming enables us to take file size out of the equation, so we can serve up super detailed visuals from models that are hundreds of megabytes in size:

Right now the feature is in testing in the US, so there’s a chance you can experience it via Android right now (with iOS planned soon). We hope to make it available widely soon, and I can’t wait to hear what you think!

AR in Google Maps can point you to your friends

This is one of the far-flung projects I’ve been glad to help support. New features (like this one that’s available on Pixel, and coming soon to iOS & Android):

When a friend has chosen to share their location with you, you can easily tap on their icon and then on Live View to see where and how far away they are–with overlaid arrows and directions that help you  know where to go.

It’s also getting smarter about recognizing landmarks:

Soon, you’ll also be able to see nearby landmarks so you can quickly and easily orient yourself and understand your surroundings. Live View will show you how far away certain landmarks are from you and what direction you need to go to get there.

Warrior dogs to get AR goggles

“Are they gonna use the Snapchat dancing hot dog to steer them or what?” — Henry Nack, age 11, bringing the 🔥 feature requests 😌

Funded by the US military and developed by a Seattle-based company called Command Sight, the new goggles will allow handlers to see through a dog’s eyes and give directions while staying out of sight and at a safe distance.

While looking through the dog’s eyes thanks to the goggle’s built-in camera, the handler can direct the dog by controlling an augmented reality visual indicator seen by the dog wearing the goggles.

Put the “AR” in “art” via Google Arts & Culture

I’m excited to see the tech my team has built into YouTube, Duo, and other apps land in Arts & Culture, powering five new fun experiences:

Snap a video or image of yourself to become Van Gogh or Frida Kahlo’s self-portraits, or the famous Girl with a Pearl Earring. You can also step deep into history with a traditional Samurai helmet or a remarkable Ancient Egyptian necklace.

To get started, open the free Google Arts & Culture app for Android or iOS and tap the rainbow camera icon at the bottom of the homepage.

Bring cultural treasures into your home in AR through Google Search

I’m pleased to be playing a very small role in making very large things, well, rather small.

Today, Google Arts & Culture has brought together a new collection to help anyone choose their perfect virtual travel with thousands of museums and cultural destinations to explore. And with the help of our partner CyArk, we’ve launched on Google Search 37 cultural heritage sites from across the world in Augmented Reality (AR). Hop from your couch and search on your mobile phone to bring the Moai statues of Ahu Ature Huki, Rapa Nui (Easter Island), the Brandenburg Gate in Germany, or the Maya pyramid of Chichén Itzá, Mexico right into your living room.

Here’s a list of landmarks you can search & explore:

  • El Castillo – Chichen Itza, Mexico
  • Brandenburg Gate, Germany
  • Ayutthaya, Thailand
  • Eim ya kyaung Temple – Bagan, Myanmar
  • Palace of Fine Arts, Mexico
  • Chacmol statue – Templo Mayor, Mexico
  • Thomas Jefferson Memorial, US
  • Lanzón – Chavín de Huántar, Peru
  • War Canoe – Waitangi Treaty Grounds, New Zealand
  • Ahu Ature Huki, Easter Island
  • Tomb of Tu Duc (Complex of Hué Monuments), Vietnam
  • Nine Dome Mosque – Bagerhat, Bangladesh
  • Shait Gumbad Mosque – Bagerhat, Bangladesh
  • Chunakhola Masjid – Bagerhat, Bangladesh
  • Church of Sveta Sofia, North Macedonia
  • Jaulian, Pakistan
  • Flanders Field American Cemetery, Belgium
  • NASA Apollo 1 Mission Memorial, US
  • Rano Rarako, Easter Island
  • Mexico City Metropolitan Cathedral, Mexico
  • Normandy American Cemetery, France
  • Ananias Chapel, Syrian Arab Republic
  • Ancient Corinth, Greece
  • Ahu Nau Nau, Easter Island
  • Lukang Longshan Temple, Taiwan
  • Temple of Apollo (Portara), Greece
  • Audiencia in Palacio Tschudi – Chan Chan, Peru
  • Palacio Tschudi – Chan Chan, Peru
  • Edinburgh Castle, UK
  • Mesa Verde, USA
  • Temple of Echmoun, Lebanon
  • Fort York National Historic Site, Canada
  • Great Mosque – Kilwa Kisiwani, Tanzania
  • Gateway of India
  • Martin Luther King Memorial
  • Lincoln Memorial