Bryan O’Neil Hughes shows off some rather eye-popping (if we may say so) technology for synthesizing texture inside a future version of Photoshop:
The demo starts with some small pieces, so if you’re short on time, jump to about the 2:50 mark (halfway point) for the more impressive stuff. I’ve been getting great results filling in missing areas around a panorama, as Bryan shows at the 4-minute mark. Full-screen viewing makes it easier to see the details.
Unbelievable!!!!!! The math behind the algorithms must be staggering.
Amazing!!!
That is amazing. What kind of crazy voodoo is this. You don’t have to sacrifice any chickens to get this to work do you?
Holy Schnikes…that’s amazing! Can’t wait !!
Some of that stuff is a lot like Akivis Retoucher.
I actually really appreciate how Adobe pays attention to popular plug-ins and then assimilates them into the main application.
Really.
Misspelled: Akvis
Amazing. Wonderful. I’m privately emailing you my Visa card details, John, so you can send me a copy of this PS.Next right away. 🙂
Fucking amazing! Please don’t tell me this won’t be in CS5.
Soon enough PS will read my mind
Ditto, great. Mouth agape.
What’s the story of how the algorithm “thinks” when it’s doing this? I’m guessing it’s something like “I’ll look around on all sides and try to figure out what you want”, but is there a better English-language description of the story?
tx, jd
Oh my GOD ! the algorithms for this must be insane ! This is really amazing.
Stunning. To paraphrase Al Pacino in the otherwise execrable Godfather III, “Every time I think I’ll be able to skip an upgrade of Photoshop, they drag me back in!”
That’s astounding!
Hats off to who created the algorithms!
Sold!
A really nice demonstration – amazing.
Closest thing I’ve seen yet for a “fix this image” tool.
I was just reading about texture synthesis a month or two ago, I can’t believe its making it into CS5! istockphoto must be shaking in their boots right now, those watermarks are history!
For more info on it works people, theres a nice wiki article and half way down an animation explaining patch based synthesis:
http://en.wikipedia.org/wiki/Texture_synthesis
Technically, Photoshop has had texture synthesis code since the Pattern Maker plug-in was introduced. It just took a long time to get the concept to work well with arbitrary textures/scenes (and it’s still not perfect, but pretty darn impressive).
Gimme! Gimme! Gimme! Wow. Gimme! Gimme! Gimme!
Amazing! i love this feature 🙂
This really shows how good Adobe is at solving customers problems and why Photoshop is still the monopoly in this segment.
AWESOME!
Interesting, how do you select samples for synthesis and how do you guarantee they are homogeneous ? Also how do you get around the problem of repetition in the synthesis areas and ensure low frequency variations are correct over the areas (like the sky)?
Wow… even added an airplane in the newly built out sky of the final panorama! How’s that happen?
I guess you mean the mouse cursor.
… where to obtain ?
DefSander
Awesome work!
Very, very cool. I can use that
Definitely awesome. I’m sure it won’t take long for me to get over the usual “unclean” feeling I get when first using new software to completely alter the reality/circumstances of the captured image.
Jaw dropping stuff this one.
It’s good to see Adobe finally working on some of the strongest existing stuff in the app, and improving and re-purposing it in clever ways. An example of other things that the healing technology, and new content analysis could be used for – texture and noise analysis in an image – and being able to unify these image characteristics for compositing tasks.
Using this new content analysis technology in other ways might be interesting – creative tools I mean, introducing random factors maybe, applied to characteristics such as color and rhythm? Whilst maybe not always practically useful for straight retouching tasks, these kinds of features would certainly make good demo video’s for CS6.
Okay, but when do we get the Jessica Alba filter?
Absolutely unbelievable. And I thought there’s not much to improve…
[Heh. That’s what the team said when they added layers in PS 3.0. –J.]
This should be great for removing watermarks on pirated images! 🙂
Seriously though, I wonder if this will work on multiple selections at the same time. For example, when restoring old photos that have pieces of emulsion missing, you can somtimes select those emulsion-less pieces pretty easy but today you still have to resort to healing/clone. If this new content-aware fill could work on the selection of those emulsion-less pieces and fill them all in at once, that would be amazing and save a lot of tedious work.
You want to remove Jessica Alba from your photos?
[That sounds like a nice problem to have. 🙂 –J.]
Me want.
Beautiful. I wondered what the end game was when the “content-aware” idea was first introduced with that Content Aware Scale (was that CS4?). It awesome to see where that ground work has led.
So I gotta ask, Is this the end game of the Content Aware tech? Or is it just warming up?
[I’m guessing that it’s just warming up. Note, though, that “Content-Aware” is kind of a generic term that’s meant to denote operations that pay attention to the content on which they’re acting, as opposed to just treating everything uniformly. In other words, this algorithm is, as far as I know, unrelated to the seam carving algorithm used in Content-Aware Scaling (introduced in CS4). –J.]
WOW! I skipped CS4 but I won’t be skipping CS5. Fantastic work!
“not much to improve…” – yes there may be a lot more room for detrimental mistakes, than genuine improvement sadly. That summarizes the worrying situation that we are in with Photoshop. As CS4 demonstrated with its more than questionable change to Adjustment Layers.
[I think you’ll like the polish we’re putting into the Adjustments panel, among many other areas. Beyond that, I’m not sure what in particular is bothering you. –J.]
On a more optimistic note, there are obviously still many highly astute and experienced imaging people, within the Photoshop Team more than capable of keeping everything on track. As the development above shows. Be interesting to see how CS5 in total pans out in this respect.
Is this an early April Fool’s? Freakin’ wow.
Well I said it would take a significant update to get me to update to Ps5. I think this qualifies.
I’m insanely impressed. Thanks for the link Jonny
Seriously, April Fools Day gets earlier and earlier every year. I call shenanigans.
OMG?! I am totally getting new version of Photoshop.. no more 5 hour works to add 10 px of width to the picture 😀
Any sufficiently advanced technology is indistinguishable from magic.
Arthur C. Clarke
How do you specify the content for content-aware fill? If there’s image data on multiple layers that I wish to use with content-aware fill, could I point the tool toward those layers?
[Generally there’s not much in the way of control: the algorithm analyzes the image & just does its thing. Maybe in the future we’ll be able to add more options for controlling the behavior, though of course that means trading away simplicity for power. Let’s see what customers say, then decide. –J.]
Very impressive. The fills truly look as though a human went in and elaborately retouched those areas – they aren’t just ‘dumb’ clones. Everyone who owns Photoshop WILL get this upgrade.
I don’t even know what to say…
Mind boggling. Great job, Photoshop dev team.
that’s just cool. for this I will delete my pirated pixelmator and switch back to a pirated photoshop!
[Nothin’ like honesty. ;-P –J.]
Now that’s more like it.
I never upgraded to CS4 because I couldn’t see any significant feature it added that I was going to use, and the interface as a Mac user was a step backwards.
This, however, this is impressive, to put it mildly. This is the kind of feature that even if CS5 DOES have an uglier interface than CS3, it would still be worth putting up with. The kind of feature that is going to make me get out my credit card to upgrade.
Frankly, it’s probably the most significant new feature (that I personally will use) in Photoshop since save for web.
TOTALLY AMAZING!
This feature is going to get a lot of photojournalists in trouble.
Hmm … where did the bird in the top left of the final extended panorama shot come from? I can’t see it anywhere in the original image.
Nice!
“let Content Aware chew through that…”
I hope that you guys consider renaming the progress bar title “Chewing Through That…”
[Oh yes–and sell Flash ad space on there: “Chewing Through That, By Doritos(TM)!”
I second the call for shenanigans.
Good God this looks JUST AWFUL.
But will it run Flash?
I assume you’re talking about the mouse cursor? I don’t see anything else in the upper-left corner after the chewing is done…
HOAX!
This feature is fake. Totally photoshopped.
😉
Really amazing.
fair play. with the compression it’s hard to make out that’s what it is but watching the video again I agree that it is the mouse cursor 🙁
What the what! Crazy.
You say “A future version of photoshop” is this a for sure feature of CS5 or do we have to wait for April 12 to find out?
This saves me no time at all because I am just that good with PS as-is. The only enhancement PS needs in my able hands is a button you click on to make money come out of the DVD burner. God, I am so good.
So, this guy — Content-Aware Phil. Is he on staff at Adobe, or a contractor that can be hired out by design firms whenever they need something fixed?
[Heh–I’ve been having that same thought for many months now. –J.]
This stinks!! It’s making us designers obsolete. JK, Now I’ll have time to sleep. No one else has to know about this, our little secrete right?
I’m sure this will be misused real quick.
Woah. The desert thing is AMAZING. Props to the Adobe dev team.
‘Content Aware’: amazing.
‘Life Aware’: gotta have this. The ability to fill in the holes in my life and cover up my mistakes, those people I wronged and most every sin on my record.
Let’s have for PS6, OK?
Thanks …
‘Content Aware’: amazing.
‘Life Aware’: gotta have this. The ability to fill in the holes in my life and cover up my mistakes, those people I wronged and most every sin on my record.
Let’s have for PS6, OK?
Thanks …
That is truly amazing.
So now I can just start with a blank image, select all, delete, and it will fill in a gorgeous landscape!
Sweet!
Isn’t this stealing the CS5 thunder just a tad?
Since I’m strictly a web designer, I’ve never forked out the cash for your products and have stuck with GIMP. I now have a compelling reason to pay money for your product, and happily.
This technique is called “In Painting” there are dozens of academic papers on in form the last ten years. It took adobe a lot of time to start incorporating it…
Probably they just bought one of the papers.
[It’s not “just inpainting.” I shared more info about the research last summer. –J.]
Hi! Layers showed up in Photoshop 2.5.
[Sorry to disagree, but it was 3.0. –J.]
I’m a trainer, and boy, we nearly genuflected in front of the product when those layers came out! And you know???…we all managed to survive without all this fancy stuff, but the awesome things we can do NOW?? Never going back!
It is in painting, but the algorithm is new and was developed in collaboration with Princeton and UW.
See:
http://www.adobe.com/technology/graphics/patchmatch.html
since the panels in CS4 are flash based, and i assume they will keep those.. then yep.. it will run flash 😉
[Just for the record, the only Flash-based panels in CS4 are those under Window->Extensions (namely Kuler & Connections). –J.]
Wow,the developers hit a home run on this feature alone.
Well now I sold for sure, the panorama was the selling point that did it for me.
The GIMP has this feature
http://o3.tumblr.com/post/470608946/photoshops-caf-content-aware-fill-unbelievable
This sound too good to be true.
Are you sure this is not an April fools joke??
This is astounding and a great reason to get upgrade to CS5. I’m absolutely sure this’ll improve workflow. :]
Sadly, an earlier poster, Pat, made a good point about what this could do in the wrong hands. Looks like artists will need to find new ways to mark their work as their own soon!
Hope this isn’t an April Fool’s Day joke!
Alien Software’s Smart Fill plugin has had this type of functionality for years.
I’m afraid that this somehow creeped me out and is going to cause me a lost night of sleep.. worrying that I’ll be deleted and cataloging pictures in my mind that need work on them.
Bella dimostrazione.
Non conosco l’inglese e quindi non ho capito la spiegazione.
Ma!!!!!!
Nella ricostruzione del paesaggio con le belle nuvole, l’ultima dimostrazione, COMPARE UN VOLATILE in alto a sinistra…
O sbaglio.
Curioso no…..
I don’t think this is CS5.
I think it might be further in the future?
Looks pretty cool, but if everything becomes automatic, what is left for the retoucher to do?
I remember Layers coming in Version 3.
The bit thing I remember of 2.5 was much improved 16 bit support.
No volatile… Mouse cursor.
Impressive . As always it would be a great tool in the experienced hands and a recipe for disaster in the newbie ones 🙂 .
The crop tool’s overlay ( thirds , golden ratio , triangle , diagonals …) is a must have . It’s already nicely implemented in Lr ( R then O to switch between overlays )
Steve, you can see repetition in the example of the dessert where the road is ‘deleted’. But that can be fixed 🙂
Overall this is truly amazing and showing once again (and for all) that Photoshop (and of course Adobe) rules.
“I’m not sure what in particular is bothering you.’
Since you asked, in detail once again …
1. having adjustment layers in a separate panel or dialog so that the panel can be moved around easily and fluidly over the image area. And taking up less valuable space in the panel well.
2. – even on top notch cards the updating of the points in the new Adjustment Panel is too slow. Curves in particular is much slower and more cumbersome to operate in the panel due to live graphics update. People who work professionally doing color correction become very fast at using the controls and shortcuts, they need to be fast.
3. The inaccessibility of the toggling of display options like histogram and baseline. These used to be simple checkboxes. Its necessary to toggle these often while working.
Way to steal PatchMatch. I guess someone was paying attention at SIGGRAPH:
http://www.cs.princeton.edu/gfx/pubs/Barnes_2009_PAR/index.php
[And I guess you haven’t been paying attention the many times I’ve mentioned Adobe’s collaboration with Princeton and other institutions. But whoa, GOTCHA, you really burned us here. The jig is up!! –J.]
By the way, to further give credit where it’s due, the brilliant minds behind this kind of stuff comes from the Advanced Technology Labs at Adobe (who, yes, often work in collaboration with outside researchers, such as the Princeton collaborators on Adobe’s PatchMatch. You can read more about them on the “People” link. I sit near a few of them, and I am very, very humbled.
Peter Baird / Adobe
OMFG!
Wow, it’s so good that in the last picture of the panorama, it puts hills and clouds in the gaps which didn’t previously exist, and invents details like a valley with a shadow over it and decides that the hill on the far right has a slope and more background behind it. Seems like this demo is either a mock up (i.e. it’s FAKED) or Photoshop actually visits the location, takes pictures of the bits that aren’t in the photo and fills them in while your progress bar is sliding along.
Besides which, the patch tool on CS3 seems to do a perfectly good job of interpolating patterns when I’m removing items. A quick drag and it perfectly fixed a gap between two walls. Apart from being an improvement on the algorithms, I don’t understand how this is new?
[The math is new, but you’re right that at a conceptual level it’s an evolution of what’s been in PS for some time. That’s partly why we were able to add it as an option (for the Spot Healing Brush) rather than create a new tool. –J.]
With their latest earnings and now this; I would buy the stock just about now.
this can’t be real.
i am a software developer.
i make neural networks and genetic algorithms.
either they have understood the nature of thought,or its an april’s fool.
i cross my fingers hard that i am wrong.
this means other image recognition software can estimate the content of invisible sections of 3d space,and make robots navigate with more common sense.
i would love to believe this is real,i just can’t.there are missing evolutionary steps of such a software.its like,fish->reptiles–mammals—>eintein.
no.
there would have been primitive versions first.
could someone confirm that they actually are using this feature?
or all we have is a video?
WOW. I just installed it (apt-get install gimp-resynthesizer) and tested. It is amazing and works pretty good! It will be now my number #1 plugin for GIMP.
Wow. That is almost too easy. What about all my years perfecting the clone tool. Ha.
But seriously, lens flares make it better. Why take those out!
Good work, Adobe. Keep rockin.
My clients think that feature’s existed since the beginning of Photoshop.
My clients think that feature’s existed since the beginning of Photoshop.
Hooray GNU troll! Let’s keep up the GIMP-is-as-good-as-Photoshop thing until RMS gives up a slobbery blow job!
Wow, truth and authenticity in the field of documentary photography is now officially over if this feature comes out for photoshop. Can’t wait to see more scandals being revealed in future New York Times editorials.
Wow, truth and authenticity in the field of documentary photography is now officially over if this feature comes out for photoshop. Can’t wait to see more scandals being revealed in future New York Times editorials.
No, this is the lightning, the thunder is still building UP!! lol
It’s been officially over since invention of the darkroom.
Nah it’s possible. This is undoubtedly the next generation of image analysis algorithms. Some of the horizon touch ups on the first photo are nothing more than the clone tool combined with a line detection algorithm. Same with the shadows.
The content aware algorithm probably checks ALL the pixels in the image and creates different histograms of pixel intensities. In english: It creates for it’s own use, a general map of lines, objects such as shadows, and gradients of color and intensity.
Thunder comes from the sound lightening makes when it breaks the sound barrier. *
oh my god.
photoshop rocks!!!!!!!!!!!!!!!!
if the military was as innovative as photoshop..we would be (..insert outrageous technology here..)
the next phase is to make this work on several progressive video frames,as many pro artists use photoshop to retouch sfx video streams.
Sorry, but this has got to be fake. I can’t believe everyone is drinking the Kool-Aid in here. I mean, lets be realistic about it. Removing the tree and putting in a sky background, OK; filling the road with plants and desert, no freaking way. How can it generate bushes and spatially orient them so it looks anything like a bunch of bushes.
If you look closely in the first manipulations, done with the healing brush, you’ll see a “Sample All Layers” radio button
You can’t steal what you own.
Two of the PatchMatch researchers (Goldman and Shechtman) work for Adobe, and a third one (Barnes) worked on PatchMatch as an intern.
-Iván Cavero Belaunde, Adobe
It’s not fake, really.
It was demoed live in front of dozens of experienced Photoshop users at NAPP’s PS 20 event.
The event video is here, the demo is towards the end (after Russell Brown’s most excellent skit), and includes footage of the original SIGGRAPH submission.
http://www.photoshopuser.com/photoshop20th
The demo by itself, audience-shot footage, is here:
http://cs5.org/?p=578
-Iván Cavero Belaunde, Adobe
And you guys realize that this one feature is going to put THOUSANDS of photo touch-up artists out of work the day it comes out (a feature that will eventually make it to Photoshop Elements and to Apple’s little competitor to PS).
[Good as the tech is, I don’t think it’ll put anyone out of a job any more than the Healing Brush did when it debuted in 2002. People said the same thing then, but one always needs skill & attention to detail to complement what the machine can do. –J.]
It isn’t generating bushes, it is copying them from the right side of the image. If you look, you can see where the repeating patterns occur in many of these videos (like the cloud patterns in the panorama).
It doesn’t make it any less impressive, this is an amazing leap in retouching. Having the system finally process more intelligently, to see where a horizon is, to see if you’re touching up along a vertical or horizontal edge, is no mean feat. I’d love to read about the decisionmaking process that Photoshop is doing when you’re selecting such large areas, but this is a fairly straightforward extension of the healing brush concept with a lot more processing power required. This is frankly the most excited i’ve been about an Adobe product since Adjustment Layers were introduced. Most recent Adobe “upgrades” have involved little more than throwing the GUI in a blender and charging for the privilege.
It isn’t magic, but it will make the life of a professional much, much better. it is a good start, but you’re still going to have to work on the image quite a bit depending on how it will be used. If you can see the source repetition on a 300×400 pixel flash video, you certainly aren’t going to be retouching magazine covers with one click.
kudos to the Photoshop team!
With respect, the “Healing Brush” was a lame duck of a tool – as has been highlighted here by Bryan in his video… with nothing near the power of this.
Doesn’t look like *any* skill is required any more. The beginning of the end methinks.
All we need are more intelligent MS-Word-esque “aesthetic wizards” in InDesign and Quark now and we may all look for a new career.
LOL. Yeah, you’re right there Gary.
LOL. Yeah, you’re right there Gary.
I am impressed someone born in the 1980’s knows what a darkroom is. The difference between now and then is that everyone today wants internet fame, and will manipulate their picture to the Nth degree to achieve it. Anyways. I think this photoshop feature has potential to do great things. Just hope it doesn’t get misused.
Some even better examples of this awesome content-aware feature:
http://www.youtube.com/watch?v=3ScWu7pG7r0
So amazing!
Are we to conclude from all the excited comments, that there’s a significant number of people who don’t know how to take a good photograph, or perhaps they’re all graphic desighners?
Content aware fill isn’t really aimed at fixing technical defects in an image. I don’t know any professional photographer or designer who hasn’t been asked “I love that photo, but could you just remove [object]?”
It is also a plain impressive technical feat, even if it eventually winds up as little more than a way for aunt mildred to touch up her vacation photos with ease.
You don’t need photoshop for that.
LOL! Good one, Jesper.
So, Gimp has has this feature for years.
[It’s not the same thing. –J.]
The demo looks great. I tried it on a scene with a dark fence and a sky with a tree behind the fence partially showing in the sky. I selected the tree and hoped that Content-Aware Fill would replace it with the sky, but no, it used the dark fence instead. This tool should have a pointer to the region to be used in the new fill to make it really usable.
Thanks for a great job on CS5.
Each time I read this, I keep hearing it in Prof. Farnsworth’s voice.
Good news, everyone! I’ve invented a device which fixes all of the mistakes in your photos.
Zajebista opcja!
Wow, really amazing stuff at work here! I hope this is streamlined straight into After Effects. If you could use an alpha/luma matte as the “selection,” what used to take hours or even days of work to rotoscope and paint would now be a simple as a rough track in MochaAE and running the matte through this plugin!
[Yeah, the AE guys have a really annoying habit of taking whatever cool thing we do & one-upping us by making it move. 🙂 –J.]
Amazing, can’t wait to use it:(
Hi J,
Impressive stuff for sure! But the panorama shot bothers me, and your reply to “Alex” is not clear. Was there a more full original photo used in a mockup for the video? Not only does a possible bird appear in the upper left, but the topographical lines and shadows added in the lower left and right are shapes not adjacent to them in the stiched photo … Apologies if I’m not advanced enough in Photoshop to understand it, but is content-aware truly creating all the changes we see with out any prompting?
So.. I think the new ‘Content Fill’ feature is amazing and great, but as someone stated jokingly about watercolor markers.. won’t this feature completely devastate the already limited copyright protection images have today?
Also, I understand everyone wants to be able to use photoshop “like the pros”, but does this feature offer too much intuition via software, transforming unremarkable digital artists into masters with a ‘monkey flips the switch’ dynamic?
So.. I think the new ‘Content Fill’ feature is amazing and great, but as someone stated jokingly about watercolor markers.. won’t this feature completely devastate the already limited copyright protection images have today?
Also, I understand everyone wants to be able to use photoshop “like the pros”, but does this feature offer too much intuition via software, transforming unremarkable digital artists into masters with a ‘monkey flips the switch’ dynamic?
I wrote about this on my blog… I am SO amazed, it is not even funny.
At first I even assumed it must be some type of joke… to taunt us. Too good to be true, you know? But it looks like the real deal.
Shocked… Just shocked.
-Dave
microstuck.blogspot.com
no, the gimp has been able to do this for 5 years now http://o3.tumblr.com/post/470608946/photoshops-caf-content-aware-fill-unbelievable
Right now this feature hasn’t been released by adobe at all and it has existed for 5 years as a plugin for gimp.
By all means, use the best tool for the job. but today, only one such tool exists.
GIMP had this for years, we show it off in our video in this article:
http://www.inatux.com/article?r=development-gimp-version-2.7-review
It’s a rip-off of GIMP’s Resynthesizer plug-in, as this video demonstrates:
http://www.youtube.com/watch?v=hLtYPmRMYRE
amazing future, photoshop is great
WOW!
I’m assuming this new feature will be released with CS5. This is way exciting and I can’t wait to try this on my own.
WOW!
I’m assuming this new feature will be released with CS5. This is way exciting and I can’t wait to try this on my own.
Is real, I use those features in GIMP for some time ;)))))
Right now I use Resynthesizer plugin in GIMP – is what Photoshop just copy paste.
check those links
http://chathuraw.wordpress.com/2010/03/29/photoshop-cs5-content-aware-fill-vs-gimp-resynthesizer/#comment-122
http://o3.tumblr.com/post/470608946/photoshops-caf-content-aware-fill-unbelievable
also you can check about Resynthesizer in Google and you will find this algorithm is to be true …
Absolutely amazing. Congratulations.
Someone finally created an “unsuck” filter …. well done 🙂
Wow this is amazing
After 2 days of testing the new feature “Content Aware Fill” i have to say is bad! Most of the times is SO BAD that in the next days..weeks people the internet will be filled with frustrated Adobe fans, who will tell you the same thing.
[Given that CS5 hasn’t been released, how exactly are you testing it? –J.]
…long live the healing brush and the patch tool!
Hey John,
My first time contacting you. I have been drawing and coloring in PS for about 8 years professionally. I just upgraded to PS CS5 from PS 7 (don’t laugh) and wanted to comment on the Eraser Icon. (I could comment on a lot of things…) I miss very much the white interior of the eraser icon; it is now greyed in and is much harder for me to read. The white suggests absence, which an eraser accomplishes. I know this is a small thing, but loyalty goes up when even the small things are just “right”. Thanks for listening. Now, if you have influence with the programs’ planners…..
thanks so much
I love it:)
I recently saw a adobe flash banner ad, which advertised Photoshop CS5 with it’s content-aware fill feature.
As I’m collection banner ads I like the most, I’d like to grab that one, but unluckily I did only see it once on “thefreedictionary.com” and missed to save it from that page.
Maybe someone know where this ad is still shown?
Any tipster could send me a mail at enases{ at }gmail.com
Thanks!