It’s pretty OT for my blog, I know, but as someone who’s been working in computer vision for the last couple of years, I find it interesting to see how others are applying these techniques.
Equipped with ultra-high definition cameras and high-powered illumination, the [Train Inspection Portal (TIP)] produces 360° scans of railcars passing through the portal at track speed. Advanced machine vision technology and software algorithms identify defects and automatically flag cars for repair.
I found myself blocked from doing anything interesting with Apple’s Reality Composer tool due to the lack of readily available USDZ-format files. My kingdom for a Lego minifig!
Therefore it’s cool to see that they’ve released a simple utility meant to facilitate conversion:
The new Reality Converter app makes it easy to convert, view, and customize USDZ 3D objects on Mac. Simply drag-and-drop common 3D file formats, such as .obj, .gltf and .usd, to view the converted USDZ result, customize material properties with your own textures, and edit file metadata. You can even preview your USDZ object under a variety of lighting and environment conditions with built-in IBL options.
In MediaPipe v0.6.7.1, we are excited to release a box tracking solution, that has been powering real-time tracking in Motion Stills, YouTube’s privacy blur, and Google Lens for several years and that is leveraging classic computer vision approaches. Pairing tracking with ML inference results in valuable and efficient pipelines. In this blog, we pair box tracking with object detection to create an object detection and tracking pipeline. With tracking, this pipeline offers several advantages over running detection per frame.
Read on for more, and let us know what you create!
Check out how it enables real objects to occlude virtual ones:
Here’s a somewhat deeper dive into the whole shebang:
The features are designed to be widely available, not requiring special sensors:
The Depth API is not dependent on specialized cameras and sensors, and it will only get better as hardware improves. For example, the addition of depth sensors, like time-of-flight (ToF) sensors, to new devices will help create more detailed depth maps to improve existing capabilities like occlusion, and unlock new capabilities such as dynamic occlusion—the ability to occlude behind moving objects.
And we’re looking for partners:
We’ve only begun to scratch the surface of what’s possible with the Depth API and we want to see how you will innovate with this feature. If you are interested in trying the new Depth API, please fill out our call for collaborators form.
Looks like a simple but perhaps compelling use of ML & AR:
Zenia encompasses the best of computer vision and machine learning. She uses motion tracking and the data from thousands of yoga lessons to analyze my movements. During the practice, Zenia provides gentle feedback and also takes care of basic safety rules.
We are excited to announce the release of BodyPix, an open-source machine learning model which allows for person and body-part segmentation in the browser with TensorFlow.js. With default settings, it estimates and renders person and body-part segmentation at 25 fps on a 2018 15-inch MacBook Pro, and 21 fps on an iPhone X.
BodyPix 2.0 has been released, including multi-person segmentation support and a new live demo!
Here’s a 2-minute concept video followed by a 6-minute on-stage demo.
Certainly looks a lot more pleasant than this classic nightmare 🙃:
Being able to place PSD layers in space & then draw a path to animate an object along it (see attached screenshot) is pretty slick. The Aero iOS app is available now, with desktop & hopefully Android to follow.
How would one go about popping the iconic ol’ gal off her background to display her in 3D? That’s just one of the numerous challenges faced by the artists who enabled seeing new & up-close angles on the piece. Check out the quick tour: