Monthly Archives: April 2018

NVIDIA’s giving me the Content-Aware Feels

Crazy to think that it’s been 8+ years since Photoshop added Content-Aware Fill—itself derived from earlier PatchMatch technology. Time & tech march onward, and new NVIDIA research promises to raise the game. As PetaPixel notes,

What’s amazing about NVIDIA’s system and the current “Content-Aware Fill” in Photoshop is that NVIDIA’s tool doesn’t just glean information from surrounding pixels to figure out what to fill holes with — it understands what the subject should look like.

Check it out:

Researchers from NVIDIA, led by Guilin Liu, introduced a state-of-the-art deep learning method that can edit images or reconstruct a corrupted image, one that has holes or is missing pixels. The method can also be used to edit images by removing content and filling in the resulting holes. Learn more about their research paper “Image Inpainting for Irregular Holes Using Partial Convolutions.”

NewImage

[YouTube] [Via John Lin]

Drone life: Point Reyes from above

I remain a rank amateur when it comes to filming with a drone, but my skills are creeping upward when I get a chance to film. Last week we visited Point Reyes & spent a bit of time exploring the famous (and now sadly half-burntPoint Reyes shipwreck. Besides taking a few shots with my DSLR, I was able to grab some fly-by footage, below.

NewImage

A few things I’ve learned:

  • I wish I’d taken a few minutes to learn about Point of Interest Mode, which you can invoke easily via the controller (see another great little tutorial on that). It would’ve made getting these orbiting shots far easier & the results much smoother.
  • They say that “Every unhappy family is unhappy in its own way,” and nearly every 360º stitching attempt with Lightroom or Camera Raw craps out in some uniquely ghoulish manner. (I’m presently gathering materials to share with my Adobe friends.) Having said that, I have a certain affection for the weird result it produced below. ¯\_(ツ)_/¯
  • The PT Gui trial seems to handle the images fine, but on principle I don’t feel like paying $125-$250 for a feature in the apps I’m already paying for.
  • Consequently I’m finding it much better to stitch panos directly on-device using the DJI Go app. (Even that doesn’t always work, sometimes stalling out for no discernible reason.) I’m also finding it impossible to load pano images back onto the SD card and stitch them in the app—so the the opportunity while you can.
  • The stitched results often (always?) fail to register as panos in Facebook & Google Photos, so I use the free Exif Fixer utility to tweak their metadata. It’s all kind of an elaborate pain in the A, but I’m sticking with this flow until I find a smoother one.
Tips, tricks, and feedback are most welcome!

 NewImage

Update: Here’s the boat from above (fullscreen):


——

Tangentially related: I shot this 360º amidst the redwoods where we camped. I stitched it in-app, then uploaded it to Google Maps so that I could embed the interactive pano (see fullscreen).

[YouTube]

Creating a 3D model of you via a simple video

Check out this neat technique in action:

Science notes,

The researchers tested the method with a variety of body shapes, clothing, and backgrounds and found that it had an average accuracy within 5 millimeters, they will report in June at the Computer Vision and Pattern Recognition conference in Salt Lake City. The system can also reproduce the folding and wrinkles of fabric, but it struggles with skirts and long hair. With a model of you, the researchers can change your weight, clothing, and pose—and even make you perform a perfect pirouette. No practice necessary.

NewImage

[YouTube]

Google puts tiny Street View cars into Hamburg’s Minatur Wunderland

Heh—this crazy project provides an extreme close-up on the incredible craftsmanship & patience of the creators of Germany’s famous miniature world. The team writes,

Street View cameras have floated on gondolas in Venice, ridden on camels in the Liwa Desert and soared on snowmobiles on Canadian slopes. But to capture the nooks and crannies in Miniatur Wunderland, we worked with our partner at Ubilabs to build an entirely new—and much smaller—device. Tiny cameras were mounted on tiny vehicles that were able to drive the roads and over the train tracks, weaving through the Wunderland’s little worlds to capture their hidden treasures.

Check out the results.

NewImage

NewImage

NewImage