Category Archives: Shit That Actually Matters

Adobe previews tools for detecting object manipulation

Back in 2011, my longtime Photoshop boss Kevin Connor left Adobe & launched a startup (see NYT article) with Prof. Hany Farid to help news organizations, law enforcement, and others detect image manipulation. They were ahead of their time, and since then the problem of “fake news” has only gotten worse.

Now Adobe has teamed up with Twitter & others Content Authenticity Initiative, and last night they previewed Project About Face, meant to help spot manipulated pixels—and even maybe reverse the effects. Check it out:

NewImage

[YouTube]

Google announces the biggest corporate purchase of renewable energy in history

Heck yeah:

This purchase is made up of a 1,600-megawatt (MW) package of agreements and includes 18 new energy deals. Together, these deals will increase our worldwide portfolio of wind and solar agreements by more than 40 percent, to 5,500 MW—equivalent to the capacity of a million solar rooftops. Once all these projects come online, our carbon-free energy portfolio will produce more electricity than places like Washington D.C. or entire countries like Lithuania or Uruguay use each year.

Our latest agreements will also spur the construction of more than $2 billion in new energy infrastructure, including millions of solar panels and hundreds of wind turbines spread across three continents. In all, our renewable energy fleet now stands at 52 projects, driving more than $7 billion in new construction and thousands of related jobs.

NewImage

Google is giving away 100,000 Home Minis to people living with paralysis

Particularly as the uncle of a little dude who uses a wheelchair, this news makes me very happy & proud:

Google  announced this morning via blog post that it has partnered with the Christopher & Dana Reeve Foundation to give away 100,000 Home Mini units to people living with paralysis. The news is designed to mark the 29th anniversary of the Americans with Disabilities Act (ADA), which was signed into law on this day in 1990.

There’s a form on Google’s site for people who qualify and their caregivers. Interested parties must live in the United States to receive a unit.

[YouTube]

The amazing Google Live Transcribe is here

Having watched my teammate Dimitri use Live Transcribe in meetings for the past year, I’m super excited to see it arrive:

[It’s] a free Android service that makes conversations more accessible through real-time captioning, supporting over 70 languages and more than 80% of the world’s population.

Here’s a deeper look into how it works.

Paul Thurrott writes,

Given my experience with my deaf son, who uses cochlear implants, lip-reading, and sign language to communicate with others, I can tell you that these apps—unlike certain misguided Microsoft accessibility efforts, like Cortana screeching during Windows Setup—address real-world problems that impact many, many people. And that they are, thus, both well-intentioned and truly useful. Bravo, Google.

[YouTube]

Accelerate machine learning through new TensorFlow Lite GPU

I’m thrilled to say that the witchcraft my team has built & used to deliver ML & AR hotness on Pixel 3, YouTube, and beyond is now available to iOS & Android developers:

For Portrait mode on Pixel 3, Tensorflow Lite GPU inference accelerates the foreground-background segmentation model by over 4x and the new depth estimation model by over 10x vs. CPU inference with floating point precision. In YouTube Stories and Playground Stickers our real-time video segmentation model is sped up by 5–10x across a variety of phones.

We found that in general the new GPU backend performs 2–7x faster than the floating point CPU implementation for a wide range of diverse deep neural network models.

A preview release is available now, with a full open source release planned for the near future.

I often note that I came here five (five!) years ago to “Teach Google Photoshop,” and delivering tech like this is a key part of that mission: enable machines to perceive the world, and eventually to see like artists & be your brilliant artistic Assistant. We have so, so far to go, and the road ahead can be far from clear—but it sure is exciting.

* One Song

“Please, please don’t come to Google and waste your time.”

I tell this to promising interview candidates. That is, I hope they come here, but it’s waaaaay too easy to fall into a velvet fog: you get free food, good money, something for your parents to brag about… but you wake up one day and realize that you’re polishing some goddamn stupid widget 9 levels deep in who-knows-what system, and you think, “Is this why I was put on earth?” This doesn’t have to happen, and indeed people often do amazing things instead—but it’s anything but guaranteed.

I always think of the amazing monologue in Walk The Line (starts around 1:30 in the clip below). If you had one song to sing before you’re dirt, are you telling me this would be it?

Now go find your song & sing the hell out of it.

Filed under “Shit That Actually Matters.”

Adobe previews new selection hotness

Wanna feel like walking directly into the ocean? Try painstakingly isolating an object in frame after frame of video. Learning how to do this in the 90’s (using stone knives & bear skins, naturally), I just as quickly learned that I never wanted to do it again. Thankfully tools like Rotobrush have come to After Effects, but like Quick Select in Photoshop, it was always pretty naive—never knowing what it was looking at.

Upon joining Google in 2014, I saw some amazing early demos of smarter techniques to isolate objects in video. While trying (unsuccessfully) to bring the tech to Google Photos, I kept hucking research paper links over the fence to my Adobe pals saying, “Just in case you’re not already looking into this—please get on it!” I always figured they were.

Smash cut to 2018. I finally get to work with those folks I met in 2014, bringing fast segmentation to Pixel 3 (powering selfie stickers, accelerating Portrait Mode) and beyond. Meanwhile Adobe is publishing their own research and showing how it might come soon (🤞) to After Effects. Check out this rad demo:

Meanwhile, if you want to try some of this hotness today, check out Select Subject—which is likely already in your copy of Photoshop!

NewImage

[YouTube 1 & 2]

“Persuasion & Control”: A dark, bracing podcast

“Like a meeting room that simply seeks to prolong the meeting”—so says writer Zeynep Tufekci of Facebook, YouTube, and other sites that seek to maximize their command of your attention. In this conversation with Sam Harris she explains, among other things…

  • how machine learning draws us towards the edges of discourse (because that’s simply what we’re most likely to click)
  • how political campaigns can (and now do) target specific people (e.g. black men in Philadelphia in 2016) in order to depress their votes
  • how marketers could detect a manic-depressive person’s manic upswing & comp him airline tickets to Vegas, knowing that he’s most vulnerable to blowing all his money
…and more. It’s fascinating, dark stuff that should give pause to all of us—especially those of us who merrily work to extract more & more insights into individuals, in order to better shape their behavior (for good, we swear…).

Sam Harris speaks with Zeynep Tufekci about “surveillance capitalism,” the Trump campaign’s use of Facebook, AI-enabled marketing, the health of the press, Wikileaks, ransomware attacks, and other topics.

Merry f’ing Christmas.

NewImage