I feel vaguely ungrateful marking the arrival of a welcome Photoshop feature by noting that Adobe first demoed it nearly four years ago—but hey, you come here, you know you’re gonna get a little salt. 😌

There’s often a lot of work to go from tech demo to robust, shipping feature (especially when targeting Photoshop’s rigorous level of quality & flexibility), and I’m sure the team has been working hard on that. In any event, I’m looking forward to trying it myself.

Check out PetaPixel’s coverage for other details & screenshots.

Invisibility cloak, engage!

From the project site:

Since Photoshop introduced the content aware fill tool, it has been familiar between several artists who used it to create different concepts. All what I had seen till now are pieces working through static images, but Zach Nader made in 2012 optional features shown, a 02:10 min video using the same tool over some commercial cars in which the texts, cars and people have been replaced by the content aware’s background. I find very interesting the glitchy movement over a constant and quiet background.

[Vimeo] [Via]

Sylvain Paris (creator of previous eye-popping tech demos) presents a sneak peek of a way to transfer the appearance of lighting in one image or video to another. Any yes, you’ll find Rainn Wilson’s interjections annoying. I recommend skipping the first 1:30. (Can anyone tell me exactly how to do that with embedded YouTube content, by the way?)

Here’s a brief look at new image-sharpening technology the Photoshop team has been developing (previously hinted at during the last Adobe MAX):

A. This guuuuuuy!

Fascinating:

Researchers at the Max Planck Institute for Informatics (MPII) have developed video inpainting software that can effectively delete people or objects from high-definition footage. The software analyzes each video frame and calculates what pixels should replace a moving area that has been marked for removal. 

More detail is on Gizmag. [Via Bruce Bullis]

Previously: Diminished Reality; Tourist-Zapping in Photoshop.

Begone, hands! New technology out of Hong Kong promises to:

allow animators to perform and capture motions continuously instead of breaking them into small increments… More importantly, it permits direct hand manipulation without resorting to rigs, achieving more natural object control for beginners.

The first half of the demo is pretty dry, but jump ahead to about 2:20 to start seeing the big wow:

[Via David Simons]

I can’t claim to have known his name, but like you I know his work: Petro Vlahos pioneered blue- and green-screen techniques & founded Ultimatte before passing away this past week at the age of 96. The BBC writes,

Mr Vlahos’s breakthrough was to create a complicated laboratory process which involved separating the blue, green and red parts of each frame before combining them back together in a certain order.

He racked up more than 35 movie-related patents and numerous Academy commendations.
By coincidence, I came across the following peek behind the scenes of The Hobbit. It bears out what Robin Shenfield from compositing firm The Mill says of Mr. Vlahos’s work: “It’s the absolute building block of all the visual effects that you see in television and movies.”

[Via]

Alyosha Efros has some ideas on cracking a particularly hard nut:

Apparently there’s a Java demo available, but it seems I long ago disabled Java. [Via Sylvain Paris]

Color me deeply skeptical, but intrigued: The BBC reports on an app that modifies the paper version of The Tokyo Shimbun in ways kids might appreciate:

“What it’s really about is something that’s been talked about for a long time, about content being presented in different ways depending on who the user is,” he said.

“It means two versions of the content – a grown-up one and the kids one. That has enormous potential. It also tackles a big gap in young readership.

This makes me oddly wistful: I’m Proust-ing out, almost smelling the newsprint & listening to the “funny papers” rattle as my dad read me cartoons, or as he’d read news & obits with a drink after work. The real obit, of course, is for the paper newspaper: I’m afraid all this will show up as a quaintly hilarious discovery that flits by on some future adult’s in-optic-nerve newsfeed. But whatever; I’m suddenly, and surprisingly, all choked up.

Adobe publishes some of its best work (e.g. tech behind Content-Aware Fill) in the academic community, rather than keeping it a trade secret, as some other big software companies do. Dan Goldman, one of the brains behind CAF, writes,

First, by encouraging publication, we make it attractive for the best minds in the business to come work in our labs – we count several former and current University professors among our ranks. Second, our researchers draw on the wealth of knowledge in the academic community as well – a great deal of our research is done in collaboration with graduate students like Connelly. And third, the rigorous demands of peer review keep us motivated to try truly new things – rather than being content to simply do all the old things better.

Check out the rest of Dan’s post for more insights into how the groundbreaking Content-Aware Fill came to be.

Buckets of inspiration, including a bunch from Adobe researchers & collaborators (HelpingHand & more):

(Now I kinda want to use the pseudonym “Laplacian Eigenfunctions.”)

It’s far from the flashiest task, but placing cuts & transitions in interview footage can be crucial to telling a story. Adobe’s Wil Li plus UC Berkeley-based collaborators Maneesh Agrawala and Floraine Berthouzoz have unveiled “a one-click method for seamlessly removing ’ums’ and repeated words, as well as inserting natural-looking pauses to emphasize semantic content.”:

To help place cuts in interview video, our interface links a text transcript of the video to the corresponding locations in the raw footage. It also visualizes the suitability of cut locations… Editors can directly highlight segments of text, check if the endpoints are suitable cut locations and if so, simply delete the text to make the edit. For each cut our system generates visible (e.g. jump-cut, fade, etc.) and seamless, hidden transitions. 

Here’s more info about the project.

Normally we work so hard to reduce motion in video (e.g. bringing the awesome Warp Stabilizer from After Effects to Premiere Pro CS6). There are cases, though (e.g. monitoring a heartbeat, or the breathing of a baby) where one wants to do just the opposite. Here’s an interesting demo:

[Via Pedro Estarque]

You know cinemagraphs, “still photographs in which a minor and repeated movement occurs”?  They can be extremely cool, but creating them is tricky.

Now Adobe researcher Aseem Agarwala & colleagues at UC Berkeley have devised “a semi-automated technique for selectively de- animating video to remove the large-scale motions of one or more objects so that other motions are easier to see.” It’s easier seen than described:

From the project site:

The user draws strokes to indicate the regions of the video that should be immobilized, and our algorithm warps the video to remove the large-scale motion of these regions while leaving finer-scale, relative motions intact. However, such warps may introduce unnatural motions in previously motionless areas, such as background regions. We therefore use a graph-cut-based optimization to composite the warped video regions with still frames from the input video; we also optionally loop the output in a seamless manner.

Our technique enables a number of applications such as clearer motion visualization, simpler creation of artistic cinemagraphs (photos that include looping motions in some regions), and new ways to edit appearance and complicated motion paths in video by manipulating a de-animated representation. We demonstrate the success of our technique with a number of motion visualizations, cinemagraphs and video editing examples created from a variety of short input videos, as well as visual and numerical comparison to previous techniques.

Brilliant use of LEDs & cameras:

[Via Rob Cantor]

Researcher Jon Brandt demos a potential new feature for searching through a large library of images by identifying images that contain the same people, backgrounds, landmarks, etc.:

A couple of years ago, Esquire shot a magazine cover using not a still camera but a high-res RED video camera. What was groundbreaking becomes commonplace, and as video capture resolution increases, so does the possibility of using stills as photos.
To make that easier, Adobe engineers & University of Washington researchers are collaborating on a method of automatically finding the best candid shots in a video clip. Check it out:

Very cool–though I continue to suspect there’s a market for auto-selecting the most ridiculous, unflattering images of one’s friends…

God that’s creepy. I want to look away… but I cannot.

Creator Kyle McDonald credits Jason Saragih’s FaceTracker library, the ofxFaceTracker addon, and openFrameworks​. [Via]

Adobe researcher Eli Shechtman & collaborators have created this excellent bit of madness:

Given a photo of person A, we seek a photo of person B with similar pose and expression. Solving this problem enables a form of puppetry, in which one person appears to control the face of another.

Now, let’s see if we can pry a webcam version out of them… [Via]

Ah–I’d been wondering what that little camera icon in the Google Images search field meant. As the company explains,

You might have an old vacation photo, but forgot the name of that beautiful beach. Typing [guy on a rocky path on a cliff with an island behind him] isn’t exactly specific enough to help find your answer. So when words aren’t as descriptive as the image, you can now search using the image itself.

Or, “Any sufficiently advanced technology is indistinguishable from magic.” — Arthur C. Clarke

A little amuse l’oeil for Saturday morning:

[Via]

At Adobe MAX last month, digital imaging researcher Sylvain Paris showed off some tech he & colleagues are cooking up in Adobe’s Boston office. Here he touches on color/tone matching between photos; more sophisticated auto-correction of color and tone (based on analyzing thousands of adjustments made by pro photographers); and image de-blurring:

Lots of other really interesting MAX sneaks are collected here.

What an interesting idea: researchers are using bokeh (lens blur), instead of sharpness, to transmit lots of data into cameras (think barcodes/QR codes on steroids). Check out the demo:

Check out the project page for more details. [Via]

Interesting tech, to say the least:

More info (in German) is on the project site. [Via Tobias Hoellrich]

Did you know that the Photoshop team has a resident theoretical physicist? If you’d like to meet him, check out next Thursday’s Silicon Valley ACM SIGGRAPH talk:

Recently we and others have gained deeper understanding of the fundamentals of the plenoptic camera and Lippmann sensor. As a result, we have developed new rendering approaches to improve resolution, remove artifacts, and render in real time. By capturing multiple modalities simultaneously, our camera captures images that are focusable after the fact and which can be displayed in multi view stereo. The camera can also be configured to capture HDR, polarization, multispectral color and other modalities. With superresolution techniques we can even render results that approach full sensor resolution. During our presentation we will demonstrate interactive real time rendering of 3D views with after the fact focusing.

See previous video: Adobe demos refocusable images.

Built at last, built at last, thank God almighty, I’ll be built at last… According to Popular Science:

Developers at the Max Planck Institute for Informatics in Saarbrücken, Germany compiled 3D scans of 120 men and women of varying sizes, merging them into a single model that can be morphed to any shape and overlaid atop original footage.
The software, called MovieReshape, builds on existing programs that track an actor’s silhouette through a scene, mapping the body into a morphable model. Using the compiled 3D scans, the program can create realistic-looking and moving body parts to the programmer’s specifications.

Check out the project site for more info. [Via Jerry Harris]

At NVIDIA’s technology conference this week, Adobe researcher Todor Georgiev demonstrated GPU-accelerated processing of plenoptic images. As Engadget puts it, “Basically, a plenoptic lens is composed of a litany of tiny “sub-lenses,” which allow those precious photons you’re capturing to be recorded from multiple perspectives.” Plenoptic image capture could open the door to easier object matting/removal (as the scene can be segmented by depth), variable perspective after capture, and more.
This brief demo takes a little while to get going, but I still think it’s interesting enough to share.