Imagine loading multi-gigabyte 3D models nearly instantaneously into your mobile device, then placing them into your driveway and stepping inside. That’s what we’ve now enabled via Google Search on Android:
Take it for a spin via the models listed below, and please let us know what you think!
Volvo: Volvo XC40, Volvo XC40 Recharge, Volvo XC60, Volvo XC90
Porsche: Porsche 911, Porsche Cayenne, Porsche Macan, Porsche Panamera, Porsche Taycan
Fiat Chrysler: Jeep Wrangler 4xE
Awesome Dad-flex: Telling your tiny, Cars-loving kids that you know Guido the forklift. 😌
Granted, it was a little confusing to explain that I knew the voice of the cartoon forklift & that he was actually a brainy Italian guy who worked at Pixar—but it worked. In any case, now Guido Quaroni—who spent 20 years at Pixar & who was always a fantastic host during Adobe customer visits—has now joined the Big Red A:
“I’ve been a customer of Adobe’s software for a number of years, and I always admired Adobe’s commitment to provide top of the line tools to creatives,” said Quaroni. “When I heard about Adobe’s renewed interest in entering into the 3D market, given how much more pervasive the consumption of 3D content is becoming, I knew it was something I wanted to be a part of. I’m excited to be joining the Adobe team to help accelerate and grow their 3D offerings for creatives worldwide.”
I remain proud to have delivered, at Guido’s urging, perhaps the most arcane feature request ever: he asked for per-layer timestamps in Photoshop so that Pixar’s rendering pipeline could discern which layers had actually been changed by artists, thereby saving a lot of rendering time. We got this done, and somehow it gives me roughly as much pleasure as having delivered a photo editor that’s used by hundreds of millions of people every month. 😌
Anyway, here’s to great things for Guido, Adobe, and 3D creators everywhere!
Psst—wanna see a multi-gig, super detailed 3D model appear in your driveway almost instantaneously?
I’m delighted to say that our work in cloud-rendered streaming 3D is paying off via this year’s virtual CES show. Per the Google Cloud blog:
As part of Fiat Chrysler’s Virtual Showroom CES event, you can experience the new innovative 2021 Jeep Wrangler 4xe by scanning a QR code with your phone. You can then see an Augmented Reality (AR) model of the Wrangler right in front of you—conveniently in your own driveway or in any open space. Check out what the car looks like from any angle, in different colors, and even step inside to see the interior with incredible details.
A bit on how it works:
The Cloud AR tech uses a combination of edge computing and AR technology to offload the computing power needed to display large 3D files, rendered by Unreal Engine, and stream them down to AR-enabled devices using Google’s Scene Viewer. Using powerful rendering servers with gaming-console-grade GPUs, memory, and processors located geographically near the user, we’re able to deliver a powerful but low friction, low latency experience.
This rendering hardware allows us to load models with tens of millions of triangles and textures up to 4k, allowing the content we serve to be orders of magnitude larger than what’s served on mobile devices (i.e., on-device rendered assets).
And to try it out:
Scan the QR code below, or check out the FCA CES website. Depending on your OS, device, and network strength, you will see either a photorealistic, cloud-streamed AR model or an on-device 3D car model, both of which can then be placed in your physical environment.
David Salesin led Adobe Research for the better part of a decade, and now that he’s at Google, he & others have been collaborating with university researchers to enable fast, fun character animation:
…or at least sufficiently indistinguishable from it.
Apple is adding a feature to help blind users detect the presence & distance of people:
Here’s how people detection works in iOS 14.2 beta – the voiceover support is a tiny bit buggy but still super cool https://t.co/vCyX2wYfx3 pic.twitter.com/e8V4zMeC5C
— Matthew Panzarino (@panzer) October 31, 2020
I’m super interested in seeing how it may help regular people generate 3D models, though it remains to be seen what they’ll then do with them:
Recording memories will never be the same.
Captured in 5 seconds with #iPhone12Pro lidar
Processed in 20 seconds with https://t.co/md5CBwvTYnDirect export to @Sketchfab: https://t.co/sZNMJHoJGi
Happy birthday Olivia! pic.twitter.com/SG9JqZBaQe
— alban denoyel (@albn) October 31, 2020
Here’s the capture flow in 3D Scanner App:
Probably needless to say, 3D model creation remains hard AF for most people, and as such it’s a huge chokepoint in the adoption of 3D & AR viewing experiences.
Fortunately we may be on the cusp of some breakthroughs. Apple is about to popularize LIDAR on phones, and with it we’ll see interesting photogrammetry apps like Polycam:
Got the #iPhone12Pro yesterday and started playing with the new lidar 3D scanner. Works great! Here is one of my first captures, made with @PolyCamAI: https://t.co/hDtg911fHO via @Sketchfab pic.twitter.com/cIHbqwgdiF
— alban denoyel (@albn) October 24, 2020
Meanwhile Adobe is working to enable 3D scanning using devices without fancy sensors. Check out Project Scantastic:
They’re also working to improve the digitization of materials—something that could facilitate the (presently slow, expensive) digitization of apparel:
I’m delighted to share that my team’s work to add 3D & AR automotive results to Google Search—streaming in cinematic quality via cloud rendering—has now been announced! Check out the demo starting around 36:30:
Here’s how we put it on via the Google Keyword blog:
Bring the showroom to you with AR
You can easily check out what the car looks like in different colors, zoom in to see intricate details like buttons on the dashboard, view it against beautiful backdrops and even see it in your driveway. We’re experimenting with this feature in the U.S. and working with top auto brands, such as Volvo and Porsche, to bring these experiences to you soon.
Cloud streaming enables us to take file size out of the equation, so we can serve up super detailed visuals from models that are hundreds of megabytes in size:
Right now the feature is in testing in the US, so there’s a chance you can experience it via Android right now (with iOS planned soon). We hope to make it available widely soon, and I can’t wait to hear what you think!
“I know it’s not the ‘woke’ thing to say, but I hope the world is enslaved by an ancient soulless sentience.” — my Lovecraft-loving weirdo friend
Heh—they’re not all creepy, to be sure, but come browse fun stuff on Google Arts & Culture, view the models in 3D, and if you’re on your phone, place them in your space via AR.
DesignTaxi writes,
Some of the creatures include the Aegirocassis, a sea creature that existed 480 million years ago; a creepy-looking ancient crustacean; and a digital remodel of the whale skeleton, which is currently in view in the National History Museum’s Hintze Hall.
Exceedingly tangentially: who doesn’t love a good coelacanth reference?
Spectacular, silly 2D/3D mayhem:
Let’s hope episode four is a gritty British version that mashes up Simon & his drawer-rings…
…with the unintelligible swagger of Don’ You Go Rounin’ Roun to Re Ro:
…In the best possible way, of course.
My mom loves to remind me about how she sweltered, hugely pregnant with me, through a muggy Illinois summer while listening to cicadas drone on & on. Now I want to bring a taste of the 70’s back to her via Google’s latest AR content.
You can now search for all these little (and not-so-little) guys via your Android or iPhone and see them in your room:
Here’s a list of new models:
