Responsive Images and Web Standards at the Turning Point

by Mat Marquis

42 Reader Comments

Back to the Article
  1. Web standards are the basis for future development. They need to be clear and consistant with existing language attributes but not necessarily focused on support now. These issues are a big deal now but I think the focus on polyfilling might be hurting the long term solution. We should be thinking about the *best* solution not the easiest to polyfill. We don’t have to use the declaration right away but it will be available to enhance our sites in a couple years. An easy polyfill should be a convenient outcome but not a goal of any spec. I know this has been discussed in the community and at length elsewhere but is this even the right place for responsive images? Using existing markup and a new image format that progressively loads up to your resolution might be a much better focus in the long run. Difficult and will take time but we need to think of specs as a foundation for the future, not a hotfix for today.
    Copy & paste the code below to embed this comment.
  2. Looking at the two syntaxes it should be pretty obvious that the RICG solution is much cleaner, more elegant, and easier to read. The WHAT WG alternative seems to have been designed by a committee of people who don’t build websites all day.
    Copy & paste the code below to embed this comment.
  3. I think markup shouldn’t be burdened with having this kind of complexity entangled into it. Imagine in a few years the list of device resolutions will be so vast that the size of these picture or image set lists will take pages and pages and make the markup even more unreadable than it is now. Why not use something along the lines of image pyramids. The details of how the browsers, web servers and the new image format respond to multi resolution devices can be contained inside a black box as it were.
    Copy & paste the code below to embed this comment.
  4. In defense of “standard way” the <img> tag is actually an specialization of the <object>, why not stick to that? Imagine that the standard non-abstract way to add an image object is something like this:
    <object data=“image-file.jpg” type=“image/jpg” width=“80” height=“20”></object> Let’s “take a step back” and create a ‘responsive’ type and pass the URLs via ?
    (WARNING: pseudo-code) <object type=“image/jpg;responsive”> </object> Then you could use such style in your CSS (non-visual clients will use the “full-file.jpg”): @media screen and (min-width: 240px) {
      foo:src {
      content: url(“really-small.jpg”);
      }
    } Ok, that was awkward, too far from the simplicity target of the new standards. Let’s take a few steps back and a few steps ahead and create a simple abstraction for this: full-file.jpg alt= Ok, let’s improve the “responsive pseudo-element” to @media screen and (min-width: 240px) {
      #foo:src {
      content: url(“really-small.jpg”);
      }
    } That leads me to the essential question: do we need a new markup since this is mainly a visual issue? I believe the solution must reside in the CSS, not in the markup at all. I know the technical implications of this: all browsers must change, this is hard… I know, but nobody said it would be easy, we’re looking for the BEST solution.
    Copy & paste the code below to embed this comment.
  5. Sorry for the mess, the “textile / safe html” parser just got away with most of my pseudo-code above… any quick tip for a newbie?
    Copy & paste the code below to embed this comment.
  6. Ok, since I was eager to hear from you… I made a gist with a little explanation (the one that could not be shown directly in the comments): https://gist.github.com/2705638 Thanks!
    Copy & paste the code below to embed this comment.
  7. I would guess one point some have against the RICG proposal is that it adds a new tag. So imho the best thing* would be to make it like: mobile.jpg
      <source src=“large.jpg” media=“min-width: 600px” >
      <source src=“large_1.5x-res.jpg” media=“min-width: 600px, »
        min-device-pixel-ratio: 1.5” >
    </img>
    Instead of the much more verbose RICG proposal:
      <source src=“mobile.jpg” >
      <source src=“large.jpg” media=“min-width: 600px” >
      <source src=“large_1.5x-res.jpg” media=“min-width: 600px, »
        min-device-pixel-ratio: 1.5” >
      mobile.jpg
    </picture>
    *  if it works in old/current browsers like I assume it will given <img > and <img></img> should be treated similarly in html.
    Copy & paste the code below to embed this comment.
  8. Code above should have been:   mobile.jpg
        <source src=“large.jpg” media=“min-width: 600px” >
        <source src=“large_1.5x-res.jpg” media=“min-width: 600px, min-device-pixel-ratio: 1.5” >
      </img> instead of:
     
        <source src=“mobile.jpg” >
        <source src=“large.jpg” media=“min-width: 600px” >
        <source src=“large_1.5x-res.jpg” media=“min-width: 600px, min-device-pixel-ratio: 1.5” >
        mobile.jpg
      </picture>
    Copy & paste the code below to embed this comment.
  9. Great article Mat. I’ve been wondering how the propose standard has been coming along for responsive images so it was great to read your article on it. A quick glance at the proposed standard makes me think that the 1x and 2x are redundant. The face-600-200@1.jpg and face-600-200@2.jpg already contain the 1 and 2 so the browser vendors could just pick up the number after the @ and determine the pixel ratios off that…or am I overlooking something?
    Copy & paste the code below to embed this comment.
  10. https://gist.github.com/2705938
    Copy & paste the code below to embed this comment.
  11. Welcome to the WHATWG! ;) I think there’s a false assumption that underpins this article: bq. “While the WHATWG’s preferences, and the web developer community’s differing preferences, certainly should be considered as we finalize a standard solution to the problem of responsive images, our highest priority must remain providing a clear benefit to our users: the needs of the user trump convenience for web developers and browser developers alike. bq. For that reason (for the sake of those who use the web), it’s critical not to cast these discussions as “us vs. them.” Standards representatives, browser representatives, and developers are all partners in this endeavor. We all serve a higher goal: to make the web accessible, usable, and delightful for all. Whatever their stance on img set or picture, I’m certain everyone involved is working toward a common goal, and we all agree that a “highest common denominator” approach is indefensible. The problem is this is not the WHATWG’s goal. And by that I mean it is not Ian Hickson’s goal. If you think Ian sees everyone as equal partners and that we should all ‘just get along’ for the greater good, you’re going to be _quite_ disappointed ;) The thing is: this is what always happens. Hickson steps on the toes of some particular community; there’s an outraged-but-well-meaning reaction with suggestions of better communication/collaboration/let’s-all-think-of-the-greater-good and it always amounts to nought. (Ask the microformats/rdfa communities.)  Why? Because Ian Hickson doesn’t care—he evaluated the suggestions, and made a decision, and that’s that. I saw the same thing a bunch of times when looking at the markup side of HTML5 for my new book “The Truth About HTML5”:http://itsninja.com/html5book/ and what everyone says about the problems with the WHATWG (read: Ian Hickson) are true. It’s time we stopped putting up with it, or naively believing that if there’s just better communication, or collaboration, or better processes, then this time (_this time!_) it will all work out. It wont. It hasn’t in the past and I doubt it will this time. “What Kyle wrote here is exactly right”:http://cssquirrel.com/blog/2012/05/15/the-egotistical-puppet-king-and-i/ . It’s time to acknowledge that the WHATWG is the problem, not the solution. We need to work around them. My suggested solution (“as I posted here on Tim Kadlec’s blog”:http://timkadlec.com/2012/05/wtfwg/comment-page-1/#comment-71598 ) is that we do what Mozilla is doing: innovate outside the WHATWG, implement it, and take the spec to the W3C. If Mozilla isn’t innovating within the WHATWG, why should we?
    Copy & paste the code below to embed this comment.
  12. The element looks good, but the word “picture” makes me feel like it represents a photo. Even though the words are similar, I feel <image> would better represent the element, and <img> would be on the way out anyways.
    Copy & paste the code below to embed this comment.
  13. One often forgotten user group seems to be getting ignored in this debate and that is the non-developers who have to write simple code when updating pages. While I might be able to get my head around a new complex set of tags, I would not enjoy trying to explain how to do this to other people in our organisation who update buyers guides and blogs and who only have the most basic HTML skills. If the WhatWG proposal goes ahead I can’t see it actually being implemented by website owners. This suggestion http://www.webmonkey.com/2012/05/use-your-head-for-a-better-way-to-serve-images/ of doing all the work once in the head makes far more sense to me and would mean that our non-developers could still be trusted to update our website when we finally go responsive.
    Copy & paste the code below to embed this comment.
  14. Syntax issues aside, it’s a real shame they are trying to make it a part of the HTML spec. Up until now breakpoint decisions never impacted the HTML, css was the place to be if you wanted to make your site responsive. Introducing a new breakpoint (that affects the images) means going back to your html to change this. That sounds like a bad (if not horrible) solution to me. It would be much nicer to see a solution that leaves the HTML alone (because it’s really a styling/performance issue, not a structural/semantic one). It seems that these days not too many people seem to care about the separation of content, style and functionality which is a real shame.
    Copy & paste the code below to embed this comment.
  15. Either of the proposals add complexity to the existing HTML5 spec, of kind that isn’t consistent to the format.
    While Mat argued against the WHATWG proposal, I think it suits better - introducing a single attribute solves a lot of problems; introducing an extra tag and confusing usage of picture and img doesn’t. Still, there is considerable code bloat that occurs.
    I am, personally, in favour of this technique [http://www.webmonkey.com/2012/05/use-your-head-for-a-better-way-to-serve-images/] Why not consider assigning default locations to serve files to different-res media?
    Copy & paste the code below to embed this comment.
  16. The syntax certainly looks more plausible and workable to me, I wonder if it would help to really focus on how we would see content authors, working with publishing tools, publishing content without a convoluted range of variables behind publishing an image. How would it work for them, never mind developers coding up these kinds of multiple source tags with various media queries? I can imagine there needs to be some sort of combination of syntax that gives a few options that degrade gracefully, and some kind of smart image detection to determine the size and resolution required/available. Anyway, I’m really not that technical, but find this all fascinating and would love to see an elegant, workable solution for the masses.
    Copy & paste the code below to embed this comment.
  17. I think we’re all missing the point with “responsive images”. Making them respond to design changes isn’t a difficult task at all, we’ve been doing that for a while now with CSS and that’s where design changes should stay, in the CSS. Breaking the presentation/structure boundary doesn’t feel like that right answer. Besides that, who really cares if image dimensions don’t match up in a responsive design? It’s not about getting the correct dimensions of the image in there, it’s about not consuming all the user’s bandwidth by loading huge (file size) images into the DOM; it’s ALWAYS been about that. I’ve seen 1000px wide images come in under 1K, so I feel like implying that there’s a direct correlation between image dimensions and file size is leading us off in a less than desirable direction. The W3C has a Network Information API to do bandwidth checks that can be used for loading more light-weight assets in over a 3G or cellular network. With Responsive design we should be looking for 3 things:
    1. Screen size for design modifications
    2. Touch capabilities for interaction changes
    3. Available bandwidth for asset management I wrote a post about this last week, I won’t link it because that’s not what this is about, you can look it up if you want. But this 3-pronged approach to responsive design has some real legs to it. How about rather than harping at the WHATWG and W3C to add in new elements that won’t have support, we get on the browser makers to get better support for the Network Information API? Thoughts? Comments?
    Copy & paste the code below to embed this comment.
  18. Hi Mat, Has the group discussed bringing the proposal to the HTML Working Group? Ian
    Copy & paste the code below to embed this comment.
  19. I believe the solution proposed fails in too many instances. The problem you are trying to solve is a dealing with a combination of limited bandwidth and patience. It is practically impossible to know the end user preference of image size as you cannot measure their bandwidth and patience level with a good level of confidence. USE CASE ISSUES Take my usage of ebay. I do not use the mobile app and instead fake my user agent so that I am delivered full resolution images in order to make an informed decision. Even if my screen resolution was low I would like zoom the photo to look for imperfections in my purchase item. The mobile version delivers low resolution images which are not good enough to make purchase decisions. SCREEN RESOLUTION Using screen resolutions is fraught with problems, such as what if they rotate their device after download to gain more space, what if they zoom your responsive image will often be a mess. POSSIBLE SOLUTION A solution to bandwidth issues is to be able to dynamically update the image so that the visible area continues to download to the appropriate device pixel level. i.e. say you have a 800X600 screen and an 8000X6000 resolution image which has been set to a width of 100%. Your request for the image should deliver a 800*600 version of the image, if the user then zooms in to part of the image, rather than simply having a pixelated view the browser should continue to download that portion of the image to the appropriate level. CURRENT REALITY
    In my own usage I find current “responsive” images to actually be more of a pain than their unresponsive counterparts. I often find myself using multiple browsers and downloading low and high resolution version and therefore using much more bandwidth than necessary in order to support my usage. FINAL THOUGHT As images load last, they can be cached and available bandwidth is expanding by the time a responsive image framework is available will it be worth the hassle trying to guess use cases and deliver responsive images? I believe working on better image compression and using CSS and SVG as appropriate is a better solution.
    Copy & paste the code below to embed this comment.
  20. I would not rule out the WHATWG proposal on the sole reason that the syntax for alternatives is cryptic. Technically speaking, it is easier to add an attribute to existing markup than it is to replace each iMG element with a set of additional ones. What the WHATWG proposal is missing though, is the flexibility and familiarity of media queries: units and ratios are clearly missing from the proposal. But it does not mean it cannot be added later on. Although the SOURCE element has become quite familiar to markup authors, I can’t help coming back to the fact that serving appropriate content is stepping on the toes of content negotiation (“source”:http://my.opera.com/karlcow/blog/2011/12/08/responsive-images-and-transparent-content-negotiation-in-http ) and therefore could be handled natively by the user-agent, using some kind of default implementation—which could be overridden with DOM events if needed. This is more or less what iOS is doing with the @2x naming convention. And if you think about it, HTML5 and related APIs work actually like that. So if we consider that responsive images are: * an opt-in feature: you need to tell which images are responsive
    * UAs provide a default implementation based on some algorithm designed by consensus Then responsive images is as simple as a boolean attribute: image.jpeg And if you think that the default implementation is not what you need, the UA would give you everything you need to replace it, for example: image.jpeg The onbeforeload event is something that doesn’t exist yet, but it would be triggered _before_ any HTTP request is sent, so that you can decide of a responsive image strategy—and it would be the responsibility of the UA to give you everything you need to do it in the event object: network throughput, output device capabilities, etc.
    Copy & paste the code below to embed this comment.
  21. Let’s try again. Responsive images: <img src=“image.jpeg” alt=”” responsive> Overriding UA content negotiation: <img src=“image.jpeg” alt=”” responsive onbeforeload=“negotiate(event)”> :)
    Copy & paste the code below to embed this comment.
  22. I don’t see what the big deal is, both the img and picture variations are hard to use and will largely be ignored by authors, since most web content is authored by software. Has anyone thought of the fact that all CMS systems worldwide are going to have to spit out various image sizes and update their WYSIWYG editors to include this new markup? How long do you figure will that take? The other issue I’m seeing, unless I’m missing something, is the seemingly hard coupling of device resolution to image size. It’s missing a lot of context. Just because my device has a high resolution doesn’t mean I’m on broadband. Even if I am on broadband, it could be expensive. Only the combination of resolution, connection speed and connection costs begins to define the user’s actual context. Therefore, I fully agree with Scott’s remark (#1). Don’t bother authors with this. Let software do it, progressively and intelligently.
    Copy & paste the code below to embed this comment.
  23. rachelreveley wrote: bq. One often forgotten user group seems to be getting ignored in this debate and that is the non-developers who have to write simple code when updating pages. Keep in mind, non-developers in most circumstances—and certainly in the circumstance of a sophisticated responsive design—will update the content via a CMS. They won’t have to craft multiple images: their design team will do that, or the CMS will do it algorhythmically. The most complicated thing a non-designer might have to do is check a few checkboxes in the CMS. :)
    Copy & paste the code below to embed this comment.
  24. As usual, Mr Keith (AKA @adactio) is the cluefullest person in the room when it comes to the sexual politics of HTML5. His “chronology of what went wrong”:http://adactio.com/journal/5474/ and evaluation of the merits of the various proposals is definitely worth reading today.
    Copy & paste the code below to embed this comment.
  25. Once again the ongoing community-based evolution of the web has provided developers the opportunity to help form the foundations for future development. The incentive here is as good as any for individual developers and designers to step forward and make a lasting improvement and contribution to web standards for years to come.
    Copy & paste the code below to embed this comment.
  26. *Gabrielso*: It turns we’re not able to add elements inside of an `img` tag. Where it’s a self-closing tag, we’ve been told by several browser folks that it can’t be made to “look ahead” for alternate sources before closing itself. I can’t claim to understand how that works on the implementor side, but we’ve been told that a number of times. The same applies to “image”, which is an alias of “img” in a surprising number of browsers.
    Copy & paste the code below to embed this comment.
  27. *corey*: Turns out we can’t use `image`. Since time immemorial, browsers have treated `image` as an alias of `img` — even many modern browsers still do this, as it turns out. If we used `image` it would be subject to all the same complications as `img`.
    Copy & paste the code below to embed this comment.
  28. *rachelreveley*: There’s a lot of conversation around that going on as we speak. It could go either way, really. `picture` wouldn’t replace common `img` tags everywhere; it would mostly be used for large hero images, like the ones seen on Apple.com. The general use would be only a few per page — and even then, having all that consolidated in the head could get pretty unwieldy. On the other hand, it would mean just as much markup down in the `body` otherwise. Matt Wilcox is a big proponant of the `head`-based approach — his “post”:http://mattwilcox.net/archive/entry/id/1091/ on the subject is well worth a read. The important thing is that we have a chance to discuss these kinds of potential improvements in a productive, meaningful way. If the WHATWG proposal were pushed through by a few key players, we may never have a chance to put these kinds of suggestions to use.
    Copy & paste the code below to embed this comment.
  29. *Luke Stevens*: Well, as you can probably tell: I’m not above starting a _little_ bit of trouble. But “equal partners” is definitely the goal, and as you said: that means finding a way to make sure we have an equal say. I believe in web standards; I do. The process may be a little skewed right now, but the more of us that get involved in the creation of the standards the less skewed the process will become. We could likely find ways to work around the WHATWG, but we’ll get more done together — we just need to have a larger presence in these discussions, and we can only get that by showing up.
    Copy & paste the code below to embed this comment.
  30. What about using HTML data attributes? Unfortunately it would require a little help via JS, but then you’re not limited to specific tags for sizing. I agree that the creation of an additional tag is redundant since we already have <img>. ex: 300x300.jpg or desktop.jpg Just a thought… I think the bigger issue is thinking ahead to the devices and screens we’ll be developing for in the near future, ex: tv, movie theaters, auto industry, etc.
    Copy & paste the code below to embed this comment.
  31. One of the points Mat drove home in his article has to do with the priority of constituencies, a founding principle of HTML5.  I was going to leave a comment here, but I got carried away: “The Unbearable Lightness of HTML5”:http://www.zeldman.com/2012/05/17/editor-vs-constituencies/ Enjoy (and feel free to disagree).
    Copy & paste the code below to embed this comment.
  32. Definitely agree that content negotiation via network protocol between client and server is where this should be handled. Neither proposals address this neatly. Webp as a new format seems to be the perfect fit in terms of image container and it has small bandwidth-friendly compression/quality. It has very low adoption so far which means backwards compatibility shouldn’t be a problem. I quite like the idea of an OnBeforeLoad event but can see the behaviour being rolled into the UA which should know from the first http-response what to expect and what to request. Quick and dirty approaches could simply inline bitmap streams via <data>
    Copy & paste the code below to embed this comment.
  33. Whether it’s a new tag since the img won’t do; I do agree that the ‘responsiveness’ bit should be treated in the CSS. Serve the smallest by default and use media queries to server larger ones as fit.
    Copy & paste the code below to embed this comment.
  34. I think “tesmond”:http://www.alistapart.com/comments/responsive-images-and-web-standards-at-the-turning-point/P10/#19 nailed it. Ideally, the best responsive solution would be to have the UA do all the heavy lifting, not the author. Only download as much of an image as is needed to fit in the space it takes up, and as makes sense with the allocated bandwidth. If that’s not technically feasible, I would prefer “guile’s suggestion”:http://www.alistapart.com/comments/responsive-images-and-web-standards-at-the-turning-point/P10/#15 over both proposals. Why would you want to have media queries throughout the document on every image, when you could set it once in the head? That’s just extra bloat and extra work.
    Copy & paste the code below to embed this comment.
  35. Wow, such a great, great clear statement of the problem. For all those commenters who think they have an easy answers to responsive images with some kind of simple css, js or html-fu,  I want to point hardy souls to Matt Wilcox’s 50-minute exegesis of the problem and the massive list of failed solutions that have come and gone. Respect: https://vimeo.com/38947881 I want to say that I wholeheartedly with Matt’s rant at the end of this video that there’s something wrong with both the “picture” tag and the “set” attribute, and the whole idea of putting what are essentially media queries in the img tag. Yes, it is easy to read, and yes it’s clear what it’s doing, but man it is brainlessly verbose. Worse, it mixes content and presentation in the most egregious way. It feels totally anti-standards. You’re dumping all your media queries into every single picture. Ugh. UGH! It’s nice that it’s low-level, but it’s just mindlessly low-level.  It’s maddening to manage, and probably also error-prone. And all-CSS solutions won’t work for the simple reason that they don’t stop browser pre-fetching. So let’s move on from that discussion please. The real problem is controlling bloat. We’re already testing for width in JS and CSS. Now we’re also going to do so in HTML? Someone please consolidate this madness. Matt’s near-genius idea is to put “breakpoint” definitions in the head: http://www.w3.org/community/respimg/2012/05/13/an-alternative-proposition-to-and-srcset-with-wider-scope/ This is by far the best idea I’ve heard so far. Flexible. Clear. Editable. Backward-compatible. Bloat-free. I hope this get looked at very seriously. Can we get some high-profile browser reps to say “yeah, this is doable!”? This is a genuine turning-point moment in the history of markup, and it really needs to be done right.
    Copy & paste the code below to embed this comment.
  36. Gabrielso beat me to the punch on recommending the object element. This has been around for over a decade and supports fallback content. e.g.
    <object type=“image/webp” data=”[src_url].webp”>
      <object type=“image/javascript;responsive” data=”[src_url].jpeg”>
      [src_url].jpg
      </object>
    </object> That said, I believe we’re trying to solve the problem with the wrong pillar of the Web. This is an task for http content negotiation, not html. I believe the ideal mark-up is as follows
    <object type=“image” data=”[url_to_generic_entity]”>fall back html</object> The user agent will send accept headers regarding preferred image formats and resolutions. The Web server will return the best suited _representation_ of that entity
    Copy & paste the code below to embed this comment.
  37. The option looks much better. I fully agree that we need consistency with the video and audio tags.
    Copy & paste the code below to embed this comment.
  38. We are just starting to use HTML5/CSS3 and jQuery exlusivly and finally don’t even care about IE8 and below. Why over complicate a feature that probably won’t be necessary in the near future. I’ll tell you why:
    1. The web is getting faster each day (did you try 4G?)
    2. Most smart CMS will reduce images on the fly using script.
    3. Did you try this: Jpeg Mini (http://www.jpegmini.com) I find it hard already to keep up with “standards”.
    Cheers
    Micha
    Copy & paste the code below to embed this comment.
  39. All of this discussion of , image supplements, JavaScript, is redundant (for the current state of the art…). The solution is to use progressive images.  And the web can be progressively upgraded (put not intended), ie:
    - enable Keep-alive and pipelining on your server.
    - and put some extra smarts into the browser (eg: use range-requests for subsequent parts of the image).
    Copy & paste the code below to embed this comment.
  40. I think this solutions just bloat HTML documents beyond recognition. We should aim for low coupling here, and defining what specific content to serve for a gazillion devices on a document that should specify the way my content is structured doesn’t cut it. I’d much prefer a solution where browsers would reliably send their capabilities back to the server as HTTP headers. I’ve been using WURFL as a way to serve images properly depending on the reported User Agent; however, we know User Agent sniffing is pretty unreliable (and a bit inefficient). Having the client send additional headers such as device width, device height, supported formats, connection speed and other info would be a way better solution, IMHO.
    Copy & paste the code below to embed this comment.
  41. Perhaps I am being naive, but what would be so terrible about using PHP to dynamically generate the required image size? @media screen and (min-width: 300px) and (max-width: 500px) {
    .box1{background: #ccc url(‘get_dynimg.php?img=pic1&size=500’) top left;}
    } designer can simply make images max size required & php will generate smaller as required.
    a simple few lines script can work for all & probably do it in chunks, so 500px img is fine for 300-500 screen, very little excess. I’m pretty sure this can be used to provide different pics depending on the screen (close up, wide angle etc) it doesn’t really require anything to be changed from normal practice other than a discrete server side script.
    Copy & paste the code below to embed this comment.
  42. Hi there, this is a great resource for the Responsive Design Knowledge Hub. Check it out here: http://goo.gl/ISjhF6
    Thanks!
    Copy & paste the code below to embed this comment.