Category Archives: Miscellaneous

Check out “Light Fields, Light Stages, and the Future of Virtual Production”

“Holy shit, you’re actually Paul Debevec!”

That’s what I said—or at least what I thought—upon seeing Paul next to me in line for coffee at Google. I’d known his name & work for decades, especially via my time PM’ing features related to HDR imaging—a field in which Paul is a pioneer.

Anyway, Paul & his team have been at Google for the last couple of years, and he’ll be giving a keynote talk at VIEW 2020 on Oct 18th. “You can now register for free access to the VIEW Conference Online Edition,” he notes, “to livestream its excellent slate of animation and visual effects presentations.”

In this talk I’ll describe the latest work we’ve done at Google and the USC Institute for Creative Technologies to bridge the real and virtual worlds through photography, lighting, and machine learning.  I’ll begin by describing our new DeepView solution for Light Field Video: Immersive Motion Pictures that you can move around in after they have been recorded.  Our latest light field video techniques record six-degrees-of-freedom virtual reality where subjects can come close enough to be within arm’s reach.  I’ll also present how Google’s new Light Stage system paired with Machine Learning techniques is enabling new techniques for lighting estimation from faces for AR and interactive portrait relighting on mobile phone hardware.  I will finally talk about how both of these techniques may enable the next advances in virtual production filmmaking, infusing both light fields and relighting into the real-time image-based lighting techniques now revolutionizing how movies and television are made.

NASA brings the in sound from way out

Nothing can stop us now
We are all playing stars…

A new project using sonification turns astronomical images from NASA’s Chandra X-Ray Observatory and other telescopes into sound. This allows users to “listen” to the center of the Milky Way as observed in X-ray, optical, and infrared light. As the cursor moves across the image, sounds represent the position and brightness of the sources.

Google Meet adds background blur—with a twist

“But what about the Web??”

I’d endlessly ask this of my old teammates, and I kept pushing to bring Google’s ML infrastructure (TensorFlow Lite, MediaPipe, etc.) and ML models (e.g. background segmentation) to everyone via browsers. Happily that work continues to bear fruit, and now the tech has come to the Web in Google Meet. This is something I haven’t seen from competitors (which rely on native apps for segmentation).

Background blur works directly within your browser and does not require an extension or any additional software. At launch, it will work on the Chrome browser on Windows and Mac desktop devices. Support for ChromeOS and Meet mobile apps will be coming soon, we’ll announce on the G Suite Updates blog when it’s available on those devices. 

This Photoshop panel is a joke. (Seriously!)

Years ago we laughed about creating an in-app Photoshop assistant called Brushy the Talking Airbrush who would, in a slack-jawed Gomer Pyle voice, intone things like “Hey, it look like yer tryin’ to retouch a photo!”

Along somewhat similarly cheeky lines comes Infinite Jokes:

“What if Photoshop could verbally judge your decisions while you were working, or tell you the best photo related puns and jokes? Infinite Jokes is just that!” reads the description. “The free light hearted panel is the perfect retouching buddy that will provide humor for hours on end.”

Yuk it up:

Google adds new tools for kids learning from home

Just in time for our boys as they level up their math skills:

When they’re stuck on a homework problem, students and parents can use Socratic and soon can use Google Lens to take a photo of a problem or equation they need help with. Socratic and Lens provide quick access to helpful results, such as step-by-step guides to solve the problem and detailed explainers to help you better understand key concepts.

Meanwhile, 3D in Search now covers a bunch of STEM-related topics:

[S]tudents can see 3D content on Search for nearly 100 STEM concepts across biology, chemistry and more using compatible Android and iOS devices. If students search for “Quantum mechanical model,” they can view a 3D atom up close and use augmented reality (AR) to bring it into their space. Check out how to use 3D for STEM concepts. 

Here’s a list of topics to try:

Glittering cutouts get their groove on

Longtime VFX stud Fernando Livschitz (see previous) has turned to 2D, making spray-painted cutouts derived from a real dancer in order to create this delightful little animation. It’s only 30s long, but the subsequent making-of minute is just as cool:

The stop-motion dancers remind me of the brilliant MacPaint animations (e.g. of Childish Gambino) from Pinot Ichwardardi, who happened to say this about low-fi tech: