Look at the Big Picture

I dread the inevitable moment during every complex responsive project when someone raises the question, What should we do about images? Lots of the things we do on the web are hard, but I reserve a special, damaged place in my delicate dev heart just for images.

The quandary of images on the multi-device web flusters me because we’re in fact trying to solve many problems at once. Existing options to solve these problems are varied, controversial, incomplete and, in many cases, not actually available to us. When asked to contribute a strategy, I lack a straightforward answer for the same reason that others do: because one doesn’t exist.

Not to disparage the admirable flurry of invention and progress! The picture element looks like it might be an actual thing, and srcset is already happening. Godspeed! But those and other building blocks need to circle overhead a while longer before we can have at them, leaving us nothing to get on board with right now.

But let’s assume that at some point not too long from now we will have new tools at our disposal, tools that allow us to define and deliver ideally optimized images to different users with different devices, connections, and contexts. OK, then what? My “do as little possible” sensibility still feels ruffled—will we successfully incorporate image wrangling into an adapted build-the-web workflow? Or are we assuming a level of oversight that might not scale?

Problem 1: Making it stronger and faster#section2

The modern, pan-device web’s relationship to images has a slightly square-peg-round-hole feel to it, and the awkward fit chafes at multiple points. There are numerous kinds of puzzles here, though they tend to fall into one of two generalized problem sets—use cases, as they’re called in some documentation about proposed image standards.

The first set of problems involves optimization, coalesced around the notion that shoving an enormous image out to a user with a small screen is both wasteful and a little bit cruel (think bandwidth costs, browser performance, and battery life).

Those architecting the future web aim to ratchet up image performance across devices in a variety of ways: cooking up new file formats, tuning compression in existing ones, rooting around in the internals of HTTP to find opportunities for speed-boosting. What’s starting to emerge are new options for today’s implementors to start considering, like the ability to handle image-resolution switching with srcset.

It’s hard. Making one chunk of the optimization puzzle fit causes weird bulges in other areas. Early responsive image techniques using JavaScript successfully provided different images in different contexts (yay!), but got in the way of browser optimizations, which caused a performance hit (sad). One vision of the future for the picture element involves the use of named (aliased) media queries in the source element—but that could cause unfortunate coupling between HTML and CSS. Neither the chicken nor the egg can go first due to the risk of messing up someone else’s chicken or egg.

Problem 2: Making it look good#section3

If solving the first set of problems helps to prevent a user from throwing her phone across the room, a second set of issues is concerned with making her feel at home in your content: using art direction to provide appropriately-composed and meaningful images within different display contexts.

Intuitively, this is an extension of responsive design philosophy: here we’re adapting not just the physical image files, but altering the content itself to fit the constraints of the browser environment—taking an active role in defining what happens.

Art direction, thus, can’t be (easily) automated. The activity entails selecting and refining meaningful visual content, which implies human intervention, potentially on a continuous basis in the case of content-rich sites. This could have considerable impact on the process and demands of producing web-ready content. If this can’t be automated, will it really get done, consistently and widely? Or will this be a niche practice, used only by teams that have ample time and motivation?

The Eagle hasn’t really landed #section4

It doesn’t help that most of the techniques referenced here don’t exist outside of the rarified world of nightly browser builds, experimental flags, and hypothetical polyfills. So we can’t throw things at the wall and see what sticks: the walls keep moving and much of the stuff we’d like to throw hasn’t really been invented yet.

We’ll likely see evolutionary movement continue through a thousand small contests, whether it’s the compression benefits of Google’s WebP format or the selective resolution optimizations allowed by srcset. In the future, we may have more ingredients available to concoct increasingly complex image-performance cocktails from the most applicable client- and server-side techniques.

Technically, things are a bit gawky and uncertain right now, though I have no doubt it’ll get figured out, eventually. The booming sounds we’re hearing are smart people blazing the way. And we’re lucky even to hear them. That the discussions are as vibrant as they are is evidence of increasing community involvement in web standards building.

Be careful what you wish for#section5

Putting aside the technical bits for a moment, will the emerging image-related standards have sticking power with those who build the web?

Some tasks in the province of optimization—like generating variants of images in different resolutions—can be automated, although given that basic, established image-optimization tactics are widely disregarded on even the biggest sites, it’s hard to say.

As for art direction, will folks have the tenacity to continuously—potentially manually—adapt images for various different display contexts? And if devices keep proliferating, how many unique image variants are we going to need? This makes my “do as little as possible/keep it simple” antenna start beeping and blinking.

We’re defining our future right now, and we face a complex tug-of-war between craft and productivity. We’ll help ourselves by realizing that an increase of available tactics poses a corresponding increase in burden for us: choosing, tuning, and implementing the right image solution.

As we encourage the adoption of standards that allow for the active application of design principles—requiring ongoing human judgement—we should make sure we’re also pushing for optimization and performance improvements that can be taken advantage of passively.

It’s not that I’m pessimistic. I’m merely wary. I want to feel confident that we’re aware of what we’re asking for, that we don’t end up overwhelming ourselves if our wishes are granted. And that means I’m rooting for a future in which doing very little can get us at least part of the way there.