Category Archives: User Interface

Chrome is reducing memory usage, adding tab search, and more

As always I’m low-key embarrassed to find this stuff exciting, but ¯\_(ツ)_/¯ The team writes,

Chrome now prioritizes your active tabs vs. everything that’s open—reducing CPU usage by up to 5x and extending battery life by up to 1.25 hours (based on our internal benchmarks).

Plus:

You can pin tabs (for those go-to pages), send tabs to your other devices and even group tabs in Chrome. This month we’re adding tab search to the toolbox.

You’ll now be able to see a list of your open tabs—regardless of the window they’re in—then quickly type to find the one you need. It’s search … for your tabs! The feature is coming first to Chromebooks, then to other desktop platforms soon.

Search has rolled out on Chrome OS & is due to come to other platforms soon.

Oh, and the “omnibox” (URL/search/dessert topping/floor wax) is learning to do new things you type in. Initial actions:

  • Clear Browsing Data – type ‘delete history’, ‘clear cache ‘ or ‘wipe cookies’
  • Manage Payment Methods – type ‘edit credit card’ or ‘update card info’
  • Open Incognito Window – type ‘launch incognito mode‘ or ‘incognito’
  • Manage Passwords – type ‘edit passwords’ or ‘update credentials’
  • Update Chrome – type ‘update browser’ or ‘update google chrome’
  • Translate Page – type ‘ translate this’ or ‘ translate this page’

AR soups up the humble car manual

The new Ram comes with an augmented reality feature for exploring one’s 700hp whip:

Hovering the camera over the steering wheel will show customers how to use the steering wheel controls or paddle shifters, while pointing at the dashboard will show infotainment functionality.

The app was developed in just three months to roll out on the 2021 Ram TRX. The wild truck will be the first vehicle to use the Know & Go app, and it will be available on other FCA vehicles down the line.

Employee-Developed Know & Go Mobile App Debuts on 2021 Ram 1500 TRX

Give FX the finger in AR

The free new app Diorama pairs with the $99 finger-worn Litho device to let you create AR movies directly inside your phone, using a selection of props & tapping into the Google Poly library:

VR Focus writes,

“Diorama will democratize the creation of special effects in the same way the smartphone democratized photography. It will allow anyone to create beautiful visual effects the likes of which have previously only been accessible to Hollywood studios,” said Nat Martin, Founder at Litho in a statement.

When combined with the Litho controller users can animate objects simply by dragging them, fine tuning the path by grabbing specific points. Mood lighting can be added thanks to a selection of filters plus the app supports body tracking so creators can interact with a scene.

Chrome gets tab groups

“Can I get that icon in cornflower blue…?”

Being a middle-aged man getting excited about tab management in a Web browser makes me a little queasy—but hey, I live in this stuff all day, so 🎉.

You can now group tabs in Chrome:

You can collapse the tab groups, and you can make the titles small:

 My pro tip is that you can use an emoji as a group name such as ❤️ for inspiration or 📖 for articles to read.

Hey, find joy where you can, amirite? 😌

After 10+ years of teasing, Microsoft’s dual-screen device arrives

Back in the day (like, when Obama was brand new in office), I was intrigued by Microsoft’s dual-screen tablet Courier concept. Check out this preview from 2009:

The device never saw production, and some of the brains behind it went on to launch the lovely Paper drawing app for iPad. Now, however, the company is introducing the Surface Duo, and I think it looks slick:

Fun detail I’d never have guessed in 2009: it runs Android, not Windows!

The prices is high ($1400 and up for something that’s not really a phone or a laptop—though something that could replace both some of the time?), and people are expressing skepticism, but we’ll see how things go. Congrats to the folks who persevered with with that interesting original concept.

Action Blocks make tasks more accessible for those with cognitive impairments

I’ve long joked-not-joked that I want better parental controls on devices, not so that I can control my kids but so that I can help my parents. How great would it be to be able to configure something like this, then push it to the devices of those who need it (parents, kids, etc.)?

Check out the gesture-sensing holographic Looking Glass

This little dude looks nifty as heck:

The Looking Glass is powered by our proprietary 45-element light field technology, generating 45 distinct and simultaneous perspectives of three-dimensional content of any sort.

This means multiple people around a Looking Glass are shown different perspectives of that three-dimensional content—whether that’s a 3D animation, DICOM medical imaging data, or a Unity project – in super-stereoscopic 3D, in the real world without any VR or AR headgear.

UntitledImage

[Vimeo]

AuraRing, a trippy ring + wristband combo gesture system

Hmm—AR glasses + smart watch (or FitBit) + ring? 🧐

VentureBeat writes,

[A] finger could be used to write legibly in the air without a touch surface, as well as providing input taps, flick gestures, and potentially pinches that could control a screened device from afar. Thanks to the magnetic sensing implementation, researchers suggest that even a visually obscured finger could be used to send text messages, interact with device UIs, and play games. Moreover, AuraRing has been designed to work on multiple finger and hand sizes.

[YouTube] [Via]

Google & Adobe team up on XD -> Flutter

I love seeing mom & dad getting along 😌, especially in a notoriously hard-to-solve area where I spent years trying to improve Photoshop & other tools:

Flutter is Google’s UI toolkit for developers to create native applications for mobile, web, and desktop, all from a single codebase. […]

XD to Flutter simplifies the designer-to-developer workflow for teams that build with Flutter; it removes guesswork and discrepancies between a user experience design and the final software product.

The plugin generates Dart code for design elements in XD that can be placed directly into your application’s codebase.

You can sign up for early access to the plug-in here.

NewImage

AR: Adobe & MIT team up on body tracking to power presentations

Fun, funky idea:

Researchers from MIT Media Lab and Adobe Research recently introduced a real-time interactive augmented video system that enables presenters to use their bodies as storytelling tools by linking gestures to illustrative virtual graphic elements. […]

The speaker, positioned in front of an augmented reality mirror monitor, uses gestures to produce and manipulate the pre-programmed graphical elements.

Will presenters go for it? Will students find it valuable? I have no idea—but props to anyone willing to push some boundaries.

Don’t nag your family. Make Google do it.

I’ve gotta give this new capability a shot:

To assign a reminder, ask your Assistant, “Hey Google, remind Greg to take out the trash at 8pm.” Greg will get a notification on both his Assistant-enabled Smart Display, speaker and phone when the reminder is created, so that it’s on his radar. Greg will get notified again at the exact time you asked your Assistant to remind him. You can even quickly see which reminders you’ve assigned to Greg, simply by saying, “Hey Google, what are my reminders for Greg?”

Google has built… Lego-scanning radar?

No, for real. The Verge writes,

What does the computer interface of the future look like? One bet from Google is that it will involve invisible interfaces you can tweak and twiddle in mid-air. This is what the company is exploring via Project Soli, an experimental hardware program which uses miniature radar to detect movement, and which recently won approval from the FCC for further study.

But yes… Legos. See what you can make of this:

[YouTube]

AR: A virtual desktop on your actual desktop?

Here’s a pretty darn clever idea for navigating among apps by treating your phone as a magic window into physical space.

You use the phone’s spatial awareness to ‘pin’ applications in a certain point in space, much like placing your notebook in one corner of your desk, and your calendar at another… You can create a literal landscape of apps that you can switch between by simply switching the location of your phone.

NewImage

[Via]

New open-source Google AI experiments help people make art

One’s differing physical abilities shouldn’t stand in the way of drawing & making music. Body-tracking tech from my teammates George & Tyler (see previous) is just one of the new Web-based experiments in Creatability. Check it out:

Creatability is a set of experiments made in collaboration with creators and allies in the accessibility community. They explore how creative tools – drawing, music, and more – can be made more accessible using web and AI technology. They’re just a start. We’re sharing open-source code and tutorials for others to make their own projects.

NewImage

[YouTube]

Power to the people: Voice UI helps a kid find autonomy

This is pretty rad:

Robbie has duchenne muscular dystrophy, which has left him able to control only his eyes, head and right thumb joint. […] Bill Weis, a retired tech worker […] set up Robbie’s bed to be controlled by voice activation. While working on the bed, Bill had an epiphany: if he can control the bed this way, why not everything else in Robbie’s bedroom universe?

Check out the story of tech + kindness + grit:

[YouTube]

Google’s new Sketch plug-in helps you pair harmonious colors & fonts

Old Man Nack would’ve killed for this back in his designer days:

As Design Taxi writes,

“Material Theming” effectively fixes a core gripe of the original “Material Design”: that virtually every Android app looks the “same,” or made by Google, which isn’t ideal for brands.

The tool is currently available on Sketch, and you can use it by downloading the “Material” plugin on the app. Google aims to expand the system regularly, and will roll out new options such as animations, depth controls, and textures, next.

NewImage

[YouTube]

UX: BMW shows off their HoloActive Touch

Help me, Bavarian Motor Works, you’re my only hope…

TechCrunch writes,

HoloActive Touch appears to float in air, and also provides actual felt, tactile feedback in response to interactions.

As for the tech used to make the interface feel somewhat physical, even though you’re just poking around in mid-air, we’ve heard it might be sourced from Ultrahaptics, a company whose whole mission is to make it possible to feel things including “invisible buttons and dials” when you want them to be tangible, and then not when you don’t.

NewImage

[YouTube]

Voice-driven photo editing: Here we go again

Four years ago Adobe showed off a prototype of voice-driven photo editing:

Now they’re back, showing a slicker but shallower (?) version of the same idea:

Well, we’ll see. Hopefully there’s a lot more to the Adobe tech. Meanwhile, I’m reminded of various VR photo-related demos. After donning a mask & shuffling around a room waving wands in the air like a goof, you realize, “Oh… so I just did the equivalent of zooming in & showing the caption?!”

Who f’ing cares?

You know what would be actually worth a damn? Let me say, “Okay, take all my shots where Henry is making the ‘Henry Face,’ then make an animated face collage made up of those faces—and while you’re at it, P-shop him into a bunch of funny scenes.” Don’t give me a novel but cumbersome rehash, gimme some GD superpowers already.

But hey, they’re making a new Blade Runner, so maybe now Ryan Gosling will edit his pics by voice, and they’ll bring back talking cameras, and in the words of Stephen Colbert, “It’s funny because nothing matters.

NewImage

[YouTube]

Google desktop radar can identify materials, body parts

Seriously (unless, of course, the UI demo is just some elaborate trolling). I can’t wait for social media to let you apply a “Facepalm” reaction by literally jamming your phone/palm against your face. Check out the demo & read on for details:

(Of course, in the current political climate I can’t help but think, “Great, I’m glad this is the critically important shit we spend our biggest brains on.”)

NewImage

NewImage

[YouTube] [Via]

UI: The beautiful interfaces of The Martian

Territory Studio nailed a tricky middle ground (futuristic but not fanciful) in crafting some great-looking interfaces for The Martian. Take a look:

Working closely with NASA, Territory developed a series of deft and elegant concepts that combine factual integrity and filmic narrative, yet are forward looking and pushing NASA’s current UI conventions as much as possible.

Territory’s plot-based graphics includes identities and visual languages for each set, and include images, text, code, engineering schematics, 3D visualisations based on authentic satellite images showing Martian terrain, weather, and mission equipment served across consoles, navigation and communication systems, laptops, mobiles, tablets, and arm screens throughout.

In all Territory delivered around 400 screens for on-set playback, most of them featuring interactive elements. With 85 screens on the NASA Mission Control set alone, a number of which were 6mx18m wall screens, there are many moments in which the graphics become a dynamic bridge between Earth and Mars, narrative and action, audience and characters.

[Vimeo] [Via]

New Photoshop shortcuts for layer visibility, locking

Having an excessive interest in keyboard shortcuts (I once wrote an edition of a book dedicate to this subject), I’m delighted to see some welcome tweaks arriving in Photoshop CC. According to Julieanne Kost’s blog:

  • Cmd-comma hides/shows the currently selected layer(s)
  • Cmd-opt-comma shows all layers
  • Cmd-slash locks/unlocks the currently selected layer(s)
  • Cmd-opt-slash unlocks all layers

(On Windows substitute Ctrl-Alt for Cmd-Opt) [Via Jeff Tranberry]

FingerSense expands a device’s touch vocabulary

If “Double knuckle knock” becomes more than, I dunno, presumably some gross phrase you’d find on Urban Dictionary, you may thank the folks at Qeexo:

FingerSense is an enhancement to touch interaction that allows conventional screens to know how the finger is being used for input: fingertip, knuckle or nail. Further, our system can add support for a passive stylus with an eraser. The technology is lightweight, low-latency and cost effective.

[Vimeo] [Via]

New iOS design resources

  • Tethr bills itself as “The last UI kit you’ll ever need” and “The Most Beautiful iOS Design Kit Ever Made.” I’ll leave that judgement to you, but at a glance it looks like some nicely assembled PSD templates.
  • You don’t actually need Photoshop to leverage these templates, either: Adobe’s Web-based Project Parfait can extract content “as 8-bit PNG, 32-bit PNG, JPG, and SVG images.”

Concept: A multitouch car UI

Hmm—I’m not sold (at all) on the discoverability of this thing, but I remain deeply eager to see someone break open the staid, hoary world of in-car electronics. (The hyped Sync system in our new Fusion is capable but byzantine & laggy. What’s waiting a second+ after button pushes between friends—besides roughly 100 feet traveled at speed?) What do you think?

[YouTube] [Via Christian Cantrell]

I got (2)99 shortcuts but just Shift ain’t one…

Transylvanian non(?)-vampire Sorin Neica has created the “Keyboard-S,” an enormous (yet thin) keyboard designed to drive Photoshop & potentially other apps. It’s sort of a Configurator panel that’s sprung right off your screen:

I have a hard time imagining it taking off, and funding on Kickstarter is pretty anemic to date, but I found the idea interesting enough to share. [Via Gary Greenwald]

New Adobe XD pub, thoughts on CS4 UI

The Adobe XD (Experience Design) team has launched INSPIRE, a new online publication in which team members can share thoughts on how, why, and what they design; gather feedback; and more.  Among the good bits:

 

 

(Apropos of nothing, both designers are among a similarly named group of "me Julies," which seems like it should be British slang if it isn’t already.)

iPhone GUI bits

  • The guys at teehan+lax have created a slick, well organized iPhone GUI PSD file.  Geoff Teehan writes, "We created our own Photoshop file that has a fairly comprehensive library of assets – all fully editable."  Nicely done! [Via Joel Eby]
  • Felix Sockwell offers a detailed walk-through of how he developed icons for the NY Times’ iPhone app.
  • Vaunted info-design expert Edward Tufte critiques iPhone interfaces in terms of their info-to-overhead ratio. [Via]

 

Marginally related at best, but too good not to share: the highly unique unboxing video for the Samsung Omnia. [Via Russell Williams]

Photoshop.next UI hints on CNET

If you’ve read this blog for any length of time, you’ve heard me talk ad nauseam about the need to better manage Photoshop’s complexity.  We need to give you the power to make the app “everything you want, nothing you don’t.”  Last week Stephen Shankland from CNET asked what kind of progress we’re making, especially in the context of the more task-based interface of Ligthroom.  I disclosed some details that appear in his article:

 

Adobe is taking a page from the Lightroom specialization playbook for Photoshop by trying to make it more customizable to specific users and tasks. But in contrast with Lightroom, company is trying to do so without sacrificing the software’s general-purpose nature, said John Nack, [principal] product manager for Photoshop.

 

“We want to make it possible to be everything you want and nothing you don’t,” Nack said. “One of the tough things has been dealing with the enormous breadth of Photoshop. We end up presenting same interface to architects as we do to Web designers as radiologists as prepress folks.”

 

To achieve that goal, Photoshop’s interface will become more open-ended and even programmable, he said.

 

“You’ll see some of the things we’ve learned about Lightroom–making things browsable and less modal–come into Photoshop,” Nack said. In other words, it’ll be easier to shift Photoshop from one task to another.

 

With a “Configurator” application that should be released by Adobe Labs within a month or two of release the next version of Photoshop, Adobe will let users create and share their own Photoshop control panels written in Adobe’s Flash programming language, Nack added. “Our goal is to make it possible for expert users to reconfigure the environment on a task-by-task basis and share those workspaces with other people. You don’t have to write code. You can knock together an interface and make it sharable.”

 

As I’ve written previously, using Flash in the Suite isn’t about slathering the UI in a bunch of blinky banner ads; rather, it’s about giving people an easy way to connect, tune, & extend their work environments.  The AIR-based Configurator app is just one way to build these UI elements–one that lets you Lego-together functionality without writing code, then easily share the output.  The tool isn’t ready to take a bow yet, but it’s coming along really nicely & I’m looking forward to showing what it can do.

 

[Related/previous: Future Photoshop UI changes]

Phil Clevenger on the Lightroom UI

Lightroom marketing manager/former Combat Photojournalist Frederick Johnson has posted an informative 12-minute interview with Phil Clevenger, Lightroom interface designer/KPT veteran.  Phil talks about the challenges of starting with a blank slate & establishing an interaction language; the pros and cons of modality; Lightroom’s unique model for applying parametric local corrections; and much more.  I fount it well worth a look.

 

(Tangentially: Both Frederick and Phil show up among Kelly Castro’s portraits (aka Project Make Mild-Mannered Software Peeps Look Hardass), now featured on the Adobe Design Center.)

Some thoughts about platform consistency

[Note: I’m motivated to write the following as I’m hearing increasing speculation about future Adobe UI changes based on what’s appeared in screenshots, the Fireworks beta, etc.  That topic deserves its own post, and I’ll work on publishing one in the next couple of days.  Until then I won’t be tackling any of those specific issues/questions.]

 

I had a rather eye-opening experience the other day.  I over heard an Adobe employee using Photoshop exclaim, “No way… they overloaded Cmd-H!”  In other words, he was surprised that pressing Cmd-H didn’t hide the application.  He was obviously A) a Mac user, B) relatively unfamiliar with Photoshop, and C) assuming that Photoshop had made a decision to go against Mac OS conventions.

 

Er, no. 🙂

 

The actual history is that Photoshop has used the Cmd-H shortcut since something close to the dawn of time (at least as far back as 1993, when I started using the app) to hide/show the current selection (the “marching ants” that go around a selection).  This convention (like essentially all PS shortcuts) is consistent between Mac and Windows, and it’s worked the same way in Illustrator for a similarly long time.

 

When Apple introduced OS X, they decided to implement some new conventions for shortcuts.  Notably, Cmd-H hides apps; Cmd-M minimizes docs to the Dock; and Cmd-~ (technically Cmd-`) cycles among open documents.  Over time the OS has appropriated more and more shortcuts that have been used by Photoshop (F9-F12 were for actions, Cmd-Space/Cmd-Opt-Space was for zooming, etc.).

 

This puts us in a tough position.  On the one hand, I totally appreciate Apple’s efforts to drive consistency across the platform.  On the other, we have to tread very carefully around keyboard shortcut changes.  Pros’ fingers dance over Adobe apps like musicians’ on instruments.  When certain things have worked a certain way for 10 or 15 years across multiple Adobe apps, you don’t just toss out those conventions and all the associated muscle memory.

 

My colleague’s comment reminded me, though, that new users don’t know or care about the history here.  For them, it just looks like Adobe is blowing off useful, consistent shortcuts, going its own way for no reason.

 

So, what do we do?  “Let me customize shortcuts,” you might say–but of course we do already (and have for years), and that doesn’t affect the default experience.  No matter what we do–change or sit tight–someone is going to be P.O.’d.

 

I think we have to take things case-by-case.  As it happens, I expect we’ll change Photoshop to use Cmd-~ to cycle among document windows.  PS already supports the Windows-standard Ctrl-Tab for this function on both platforms, and by honoring both conventions we can offer cross-platform consistency.  This move will undoubtedly frustrate people who rely on Cmd-~ for displaying the composite channel, but we’ll do our best to ease the pain.  Remapping Cmd-H and Cmd-M are a progressively tougher sell for me, given the importance of selections & Curves in PS.  Note, however, that on the Mac by default Photoshop assigns Cmd-Ctrl-H to hiding & Cmd-Ctrl-M to minimizing–i.e. the standard conventions + the Ctrl key.

 

I mention all this in order to shed some light on the tricky issues we face with the Photoshop & other Adobe tools.  No one I know here views OS conventions as unimportant; on the contrary, they’re always among the first issues considered.  It’s just that we have to weigh them against possible disruptions to user habits and workflows, and against the user benefits of consistency between applications and platforms.

[PS–I know people are eager to hear more & to discuss the application frame idea, etc. As I say, I plan to post plenty of detail shortly. (In other words, please don’t fill the comments with tons of questions/rants just yet. :-)) More to come… –J.]

Flash-based multitouch coolness

Christian Moore & the folks at the NUI Group have created Lux, an open-source framework for creating multitouch-savvy applications.  Check out the video demo & a short interview with Christian on how they’ve used Flash to prototype a very cool implementation.  I’d love to see it updated to take advantage of the GPU hardware acceleration in the upcoming Flash Player 10 (just posted in preview form on Adobe Labs).  Oh, and how about this running in a Smart Object on the Photoshop canvas?  (Hey, I’m just thinking aloud, not dropping any near-term.) [Via Jerry Harris]

 

Interesting related bits:

  • Gizmodo features a short recent interview with Jeff Han, the guy whose multitouch work really lit a fire under the whole area two years ago.
  • Macworld’s Dan Frakes provides a video tour of MultiClutch, a free utility for extending the multitouch features in the latest MacBook Pro & MacBook Air notebooks.  (I was bummed to discover that my wife’s brand new MacBook doesn’t offer the same support.  She’s just happy to have two-finger scrolling, something missing from her deceased PowerBook.)
  • I need to pull together a category for multitouch; in the meantime, past interesting bits are here.

Wiimote Hacks + Photoshop

Photographer* Mike Hill passed along a link to crafty hacker Johnny Lee’s hacks to enable, among other things, whiteboarding on the cheap in Photoshop, thanks to a modified Nintendo Wii controller.  PS is shown only in passing, but it’s still fun to see.

This of course makes me think of the Flash-based, Wiimote-powered multi-user painting system created by BLITZ Agency for Adobe MAX last year (details).  It may sound frivolous, but I still like to get my little wheels turning about how to cross-pollinate this kind of Web-flavored coolness with our desktop apps.  I still want creation experiences that can feel more like this, and less like poking sliders and knobs.

* and creepily faithful Michael McDonald imitator