Look at the Big Picture

I dread the inevitable moment during every complex responsive project when someone raises the question, What should we do about images? Lots of the things we do on the web are hard, but I reserve a special, damaged place in my delicate dev heart just for images.

Article Continues Below

The quandary of images on the multi-device web flusters me because we’re in fact trying to solve many problems at once. Existing options to solve these problems are varied, controversial, incomplete and, in many cases, not actually available to us. When asked to contribute a strategy, I lack a straightforward answer for the same reason that others do: because one doesn’t exist.

Not to disparage the admirable flurry of invention and progress! The picture element looks like it might be an actual thing, and srcset is already happening. Godspeed! But those and other building blocks need to circle overhead a while longer before we can have at them, leaving us nothing to get on board with right now.

But let’s assume that at some point not too long from now we will have new tools at our disposal, tools that allow us to define and deliver ideally optimized images to different users with different devices, connections, and contexts. OK, then what? My “do as little possible” sensibility still feels ruffled—will we successfully incorporate image wrangling into an adapted build-the-web workflow? Or are we assuming a level of oversight that might not scale?

Problem 1: Making it stronger and faster#section1

The modern, pan-device web’s relationship to images has a slightly square-peg-round-hole feel to it, and the awkward fit chafes at multiple points. There are numerous kinds of puzzles here, though they tend to fall into one of two generalized problem sets—use cases, as they’re called in some documentation about proposed image standards.

The first set of problems involves optimization, coalesced around the notion that shoving an enormous image out to a user with a small screen is both wasteful and a little bit cruel (think bandwidth costs, browser performance, and battery life).

Those architecting the future web aim to ratchet up image performance across devices in a variety of ways: cooking up new file formats, tuning compression in existing ones, rooting around in the internals of HTTP to find opportunities for speed-boosting. What’s starting to emerge are new options for today’s implementors to start considering, like the ability to handle image-resolution switching with srcset.

It’s hard. Making one chunk of the optimization puzzle fit causes weird bulges in other areas. Early responsive image techniques using JavaScript successfully provided different images in different contexts (yay!), but got in the way of browser optimizations, which caused a performance hit (sad). One vision of the future for the picture element involves the use of named (aliased) media queries in the source element—but that could cause unfortunate coupling between HTML and CSS. Neither the chicken nor the egg can go first due to the risk of messing up someone else’s chicken or egg.

Problem 2: Making it look good#section2

If solving the first set of problems helps to prevent a user from throwing her phone across the room, a second set of issues is concerned with making her feel at home in your content: using art direction to provide appropriately-composed and meaningful images within different display contexts.

Intuitively, this is an extension of responsive design philosophy: here we’re adapting not just the physical image files, but altering the content itself to fit the constraints of the browser environment—taking an active role in defining what happens.

Art direction, thus, can’t be (easily) automated. The activity entails selecting and refining meaningful visual content, which implies human intervention, potentially on a continuous basis in the case of content-rich sites. This could have considerable impact on the process and demands of producing web-ready content. If this can’t be automated, will it really get done, consistently and widely? Or will this be a niche practice, used only by teams that have ample time and motivation?

The Eagle hasn’t really landed #section3

It doesn’t help that most of the techniques referenced here don’t exist outside of the rarified world of nightly browser builds, experimental flags, and hypothetical polyfills. So we can’t throw things at the wall and see what sticks: the walls keep moving and much of the stuff we’d like to throw hasn’t really been invented yet.

We’ll likely see evolutionary movement continue through a thousand small contests, whether it’s the compression benefits of Google’s WebP format or the selective resolution optimizations allowed by srcset. In the future, we may have more ingredients available to concoct increasingly complex image-performance cocktails from the most applicable client- and server-side techniques.

Technically, things are a bit gawky and uncertain right now, though I have no doubt it’ll get figured out, eventually. The booming sounds we’re hearing are smart people blazing the way. And we’re lucky even to hear them. That the discussions are as vibrant as they are is evidence of increasing community involvement in web standards building.

Be careful what you wish for#section4

Putting aside the technical bits for a moment, will the emerging image-related standards have sticking power with those who build the web?

Some tasks in the province of optimization—like generating variants of images in different resolutions—can be automated, although given that basic, established image-optimization tactics are widely disregarded on even the biggest sites, it’s hard to say.

As for art direction, will folks have the tenacity to continuously—potentially manually—adapt images for various different display contexts? And if devices keep proliferating, how many unique image variants are we going to need? This makes my “do as little as possible/keep it simple” antenna start beeping and blinking.

We’re defining our future right now, and we face a complex tug-of-war between craft and productivity. We’ll help ourselves by realizing that an increase of available tactics poses a corresponding increase in burden for us: choosing, tuning, and implementing the right image solution.

As we encourage the adoption of standards that allow for the active application of design principles—requiring ongoing human judgement—we should make sure we’re also pushing for optimization and performance improvements that can be taken advantage of passively.

It’s not that I’m pessimistic. I’m merely wary. I want to feel confident that we’re aware of what we’re asking for, that we don’t end up overwhelming ourselves if our wishes are granted. And that means I’m rooting for a future in which doing very little can get us at least part of the way there.

7 Reader Comments

  1. There is no easy answer, clearly. What we can accomplish technically always has to be weighed with many constraints and I love that you’ve brought attention to this particular one. It’s always a delicate dance to find the right solution not only for the persons using the sites we build, but also for the people who end up maintaining them. It’s unfortunate that these constraints don’t always align and sometimes compete. I’m grateful for the additional tools in the toolbox, though.

  2. Thank you for writing this, it articulates a lot of the wariness I feel right now.

    While very respected and influential designers and standards experts are telling us how things should be, it’s very hard to keep up in the wake of their work.

    It’s a really exciting time to design and develop for the web, but there is almost too much for any one individual to get to grips with – responsive images encapsulates a lot of this in just one problem.

    It’s easy to feel almost dis-empowered by the sheer range of “best practices” being espoused every day.

    I’m still optimistic about the future of web design and development, but like you, sometimes think that doing less (the right kind of less) can achieve more.

  3. Hi,
    I’m experimenting with thoses responsive pictures problems since some months…

    The solution I choosed for my illustration website is to use the great Picturefill library (v2) : it works today (with some few limitations).

    For art direction, the best way I found is to use a node and graph image editor, like the one in Blender. This allow to set up a graph once for all your sizes, and then reuse it infinitelly with just some tweakings for selecting the good crops for an image, and hit the render button.

    More explanation on the way it work here .
    Mikado is no more an active project, but others are taking shape, like imgflo.

    My website (really young, experimental for now, alpha status, do not suport anything bellow IE10) is here : http://nylnook.com

    I may give a conference at Libre Software Meeting in July to explain this workflow, but the program isn’t published yet.

  4. Good article. I share your anguish when dealing with and thinking of techniques for future image handling. I think one option could be to put a standard on device sizes and resolutions. As web developers we have standards that we must work within the confines of when creating new applications and websites. Why can’t the developers and designers of the hardware work within these same standards?

  5. Fantastic article. I’ve always encouraged auto-compression techniques, but never endorse auto-crops. For a semi-auto crop tool for your art directors, take a look @sizzlepig . We took 8 hours of PhotoShop work down to 20 minutes for our Art Directors!

  6. I’ve been trolling around this topic for about 2 weeks now… tinker with, and testing just about everything that looked like a valid or promising solution. Well, the most convincing, as it’s pretty future proof in that you don’t need to change your current image tags/markup or create more/alternate image files AT ALL, so it can be added/removed with ease, is called ADAPTIVE-IMAGES (dot) you know the rest. NO, it’s not mine… LOL. As with all, there’s a downfall, it ONLY works with images hosted on the same domain/server. It will get you UP TO SPEED with good FUNCTIONALITY plus the benefit of EASILY TRASHING and/or REPLACING it when something better comes along in the future. You can’t ask for much more. I’m going to write a blog post about it, and add some things that we coupled it with to make it work WONDERS. I’m not a big commenter, but I felt like sharing this one.

  7. By the way, the above mentioned also won’t REPLACE the image with a clear .GIF. I saved a few of those solutions; which is what Picture Fill (mentioned by “Camille”) and a few others do, to possibly use them for image SECURITY purposes instead; to prevent right-click saving of the images, i.e., they’ll get the clear .GIF instead.

Got something to say?

We have turned off comments, but you can see what folks had to say before we did so.

More from ALA

Nothing Fails Like Success

Our own @zeldman paints the complicated catch-22 that our free, democratized web has with our money-making capitalist roots. As creators, how do we untangle this web? #LetsFixThis