Adapting to Input

Jeremy Keith once observed that our fixed-width, non-responsive designs were built on top of a consensual hallucination. We knew the web didn’t have a fixed viewport size, but we willfully ignored that reality because it made our jobs easier.

The proliferation of mobile devices forced us into the light. Responsive web design gave us the techniques to design for the rediscovered reality that the web comes in many sizes.

And yet there is another consensual hallucination—the idea that desktop equals keyboard and mouse, while phones equal touch.

It’s time to break free of our assumptions about input and form factors. It’s time to reveal the truth about input.

Four truths about input#section2

1. Input is exploding — The last decade has seen everything from accelerometers to GPS to 3D touch.
2. Input is a continuum — Phones have keyboards and cursors; desktop computers have touchscreens.
3. Input is undetectable — Browser detection of touch‚ and nearly every other input type, is unreliable.
4. Input is transient — Knowing what input someone uses one moment tells you little about what will be used next.

Being adaptable#section3

In the early days of mobile web we created pitfalls for ourselves such as “mobile context.” We’ve since learned that mobile context is a myth. People use their phones everywhere and for any task, “especially when it’s their only or most convenient option.”

When it comes to input, there is a danger of making a similar mistake. We think of a physical keyboard as being better suited to complex tasks than an onscreen keyboard.

But there are many people whose primary access to the internet is via mobile devices. Those same people are comfortable with virtual keyboards, and we shouldn’t ask them to switch to a physical keyboard to get the best experience.

Even for those of us who spend our days on computers, sometimes a virtual keyboard is better. Perhaps we’re on a plane that has started to descend. In that moment, being able to detach a keyboard and work on a touchscreen is the difference between continuing our task or stowing our laptop for landing.

So who are we to judge what input is better? We have no more control over the input someone uses than we do the size of their screen.

Becoming flexible#section4

Confronting the truth about input can be overwhelming at first. But we’ve been here before. We’ve learned how to design for a continuum of screen sizes; we can learn how to adapt to input—starting with these seven design principles.

Design for multiple concurrent inputs#section5

The idea that we’re either designing for desktop-with-a-mouse or touch-on-mobile is a false dichotomy. People often have access to multiple inputs at the same time. Someone using a Windows 10 laptop or a Chromebook Pixel may be able to use the trackpad and touchscreen concurrently.

There are many web pages that detect touch events and then make incorrect assumptions. Some see the touch events and decide to deliver a mobile experience regardless of form factor. Others have different branches of their code for touch and mouse and once you’re in one branch of the code, you cannot switch to the other.

At minimum, we need to ensure that our web pages don’t prevent people from using multiple types of input.

Ideally, we would look for ways to take advantage of multiple inputs used together to create better experiences and enable behavior that otherwise wouldn’t be possible.

Make web pages that are accessible#section6

When someone uses a remote control’s directional pad to interact with a web page on a TV, the browser sends arrow key events behind the scenes. This is a pattern that new forms of input use repeatedly—they build on top of the existing forms of input.

Because of this, one of the best ways to ensure that your web application will be able to support new forms of input is to make sure that it is accessible.

The information provided to help assistive devices navigate web pages is also used by new types of input. In fact, many of the new forms of input had their beginnings as assistive technology. Using Cortana to navigate the web on an Xbox One is not so different than using voice to control Safari on a Mac.

Design for the largest target size by default#section7

A mouse is more precise than our fingers for selecting items on a screen. Buttons and other controls designed for a mouse can be smaller than those designed for touch. That means something designed for a mouse may be unusable by someone using a touchscreen.

However, something designed for touch is not only usable by mouse, but is often easier to select due to Fitts’s Law, which says that “the time to acquire a target is a function of the distance to and size of the target.”

Plus, larger targets are easier for users with lower dexterity, whether that is a permanent condition or a temporary one caused by the environment. At the moment, the largest target size is touch, so this means designing touch first.

As Josh Clark once said, “when any desktop machine could have a touch interface, we have to proceed as if they all do.”

Design for modes of interaction instead of input types#section8

Gmail’s display density settings illustrate the benefit of designing for user interaction instead of input types.

By default, Gmail uses a comfortable display density setting. If someone wants to fit more information on the screen, they can switch to the compact display density setting.

It so happens that these two settings map well to different types of input. The comfortable setting is touch-friendly. And compact is well suited for a mouse.

But Gmail doesn’t confine these options to a particular input. Someone using a touchscreen laptop could choose to use the compact settings. Doing so sacrifices the utility of the laptop’s touchscreen, but the laptop owner gets to make that choice instead of the developer making it for her.

Vimeo made a similar choice with their discontinued feature called Couch Mode. Couch Mode was optimized for the 10ft viewing experience and supported remote controls. But there was nothing that prevented someone from using it on their desktop computer. Or for that matter, using the standard Vimeo experience on their TV.

In both cases, the companies designed for use cases instead of a specific form factor or input. Or worse, designing for a specific input inferred from a form factor.

Abstract baseline input#section9

When we’re working on responsive web designs at Cloud Four, we’ve found that the labels “mobile,” “tablet,” and “desktop” are problematic. Those labels create images in people’s minds that are often not true. Instead, we prefer “narrow,” “wide,” “tall,” and “short” to talk about the screens we’re designing for.

Similarly, words like “click” and “tap” betray assumptions about what type of input someone might use. Using more general terms such as “point” and “select” helps prevent us from inadvertently designing for a particular input.

We should also abstract baseline input in our code. Mouse and touch events are entirely different JavaScript APIs, which makes it difficult to write applications that support both without duplicating a lot of code.

The Pointer Events specification normalizes mouse, touch, and stylus events into a single API. This means for basic input, you only have to write your logic once.

Pointer events map well to existing mouse events. Instead of mousedown, use pointerdown. And if you need to tailor an interaction to a specific type of input, you can check the pointerType() and provide alternate logic—for example, to support gestures for touchscreens.

Pointer Events are a W3C standard and the jQuery team maintains a Pointer Events Polyfill for browsers that don’t yet support the standard.

Progressively enhance input#section10

After baseline input has been wrangled, the fun begins. We need to start exploring what can be done with all the new input types available to us.

Perhaps you can find some innovative uses for the gyroscope like Warby Parker’s product page, which uses the gyroscope to turn the model’s head. And because the feature is built using progressive enhancement, it also works with mouse or touch.

The camera can be used to scan credit cards on iOS or create a photo booth in browsers that support getUserMedia. Normal input forms can be enhanced with the accept attribute to capture images or video via the HTML Media Capture specification:

<input type="file" accept="image/*">
<input type="file" accept="video/*;capture=camcorder">
<input type="file" accept="audio/*;capture=microphone">

Make your forms easier to complete by ensuring they work with autofill. Google has found that users complete forms up to 30 percent faster when using autofill. And keep an eye on the Payment Request API, which will make collecting payment simple for customers.

Or if you really want to push the new boundaries of input, the Web Speech API can be used to enhance form fields in browsers that support it. And Physical Web beacons can be combined with Web Bluetooth to create experiences that are better than native.

Make input part of your test plans#section11

Over the last few years, test plans have evolved to include mobile and tablet devices. But I have yet to see a test plan that includes testing for stylus support.

It makes intuitive sense that people check out faster when using autofill, but none of the ecommerce projects that I’ve worked on have verified that their checkout forms support autofill.

We need to incorporate input in our test plans. If you have a device testing lab, make input one of the criteria you use to determine what new devices to purchase. And if you don’t have a device testing lab, look for an open device testing lab near you and consider contributing to the effort.

The way of the web#section12

Now is the time to experiment with new forms of web input. The key is to build a baseline input experience that works everywhere and then progressively enhance to take advantage of new capabilities of devices if they are available.

With input, as with viewport size, we must be adaptable. It is the way of the web.