A long, long time ago the “lowsrc”:http://lists.w3.org/Archives/Public/www-html/2002Nov/0136.html attribute would handle something very similar to this. It would be nice to have it back again to quickly load an image on any browser, then once everything is in place start working on high res if possible. A non-js implementation would be ideal, as you wrote towards the end of your post better fallbacks and conditional loading would be wonderful. Some other options for responsive loading were discussed on “quirksmode”:http://www.quirksmode.org/blog/archives/2011/09/responsive_desi.html ,
Copy & paste the code below to embed this comment.
John Bertucci
I think this idea of a element.
One thing that caught my eye was the repeated idea of:
<source src=“mobile.jpg”>
mobile.jpg
Wouldn’t it be nicer to simply use [img]big-img.jpga4SRO0xrVj8VBqz7kAMQ96V0hvUt5xsBtypography_img_src_end media=“min-width:800px”> as the standard image so you only need to declare it once and add alternative images?
[img]medium.pnga4SRO0xrVj8VBqz7kAMQ96V0hvUt5xsBtypography_img_src_end media=“min-width:600px;”> has been a closed tag forever, why not just allow the tag to be opened and use a format like.
I’ve looked at it in a similar way to @steveu in comment 17. I’ve experimented with loading the small image for all browsers (sized up if necessary) and replacing it with the larger image via javascript as needed. As you mention, in high-bandwidth situations users get a flash of the blurry image, which resolves very quickly to the high-res image if the bandwidth is decent. It’s maybe not the most ideal (I like the new markup proposal) but it works adequately in every case.
In my mind, this approach has the added advantage of a super-fast initial display for everyone (including desktops), since they only have to download the teeny images at first. From what I’ve seen, many people will be okay with a progressive image load if it makes the site faster.
The <img> tag can’t be opened up because of the way legacy browsers treat it as a self-closing tag.
<img>
more stuff
</img>
The </img> tag would be treated as another img element in the DOM. And none of the contents between the <img> and the </img> would be considered as child elements. And a </img> with no src would hurt performance. See http://www.nczonline.net/blog/2009/11/30/empty-image-src-can-destroy-your-site/
It is becoming clear to me that “responsive” design leaves all layout choices in the hands of the designers (where it should be) and “adaptive” requires more consultation with the developers, and the best result will probably be a combination of them. I would like to see the web change to offering as much control over layout and content to the front end developers as possible, but we’re not there yet.
For a simple page that only contains content you want to display anywhere, if the images can be delivered as CSS background images then surely @media queries would be way to go, specifying a different image depending on screen size.
If the design has been created in that quaint, print-focussed, Photoshop-a desktop-design-first approach, then a mobile site may have to be assembled with a controller selecting a different set of models and views depending on the user agent.
Right now I’m advocating adaptive techniques as a fall back, and they are often the only way to effectively “mobilize” an old-fashioned web site reasonably quickly. The benefit of this is that the client will soon want their desktop site to be as efficient and user-friendly as the mobile version.
I’d strongly encourage everyone to read “Jason Grigg’s article on responsive images”:http://www.cloudfour.com/responsive-imgs/ (which Mat linked to.)
It explains a lot of the difficulties that you have to work around in order to develop a responsive image solution. By far the worst one is the “First page load problem”. If you want to use JavaScript to determine the capabilities of the user’s browser, you can’t do that before you send your first page’s content. And you can’t try to use some blocking JS in the <head> that determines capabilities before the body is parsed”¦ because of the new techniques Mat mentioned where they pre-fetch images before building up the DOM of the body. And you can’t use data-URIs and “blank images” because “a <img> with no src is not actually blank”:http://www.nczonline.net/blog/2009/11/30/empty-image-src-can-destroy-your-site/ .
Which means on the first connection, you don’t know anything about the browser, so you can’t send “the right image” to it. If you want to try to figure out some of the capabilities of the browser before you send the first page of data, you have to rely on UA sniffing. UA sniffing is fragile and requires constant updating of your UA database.
So, yes, you can build server-side solutions into a CMS, but that UA-sniffing code will never be as robust as a native HTML solution could be. Personally, I’d like to try to build a server-side solution that uses a Future Friendly markup with the element.
DavidSparks: Yeah, I’m not 100% married to “picture,” to represent the new element. “Poster” would make more sense, but it’s a term already in use and that might trip people up. Maybe “art?” Stuff we can nail down once we have something working, I figure.
In terms of the media attributes, you could still specify media=“screen” and the like in there—you can use any combination of media queries, the same as you would in your CSS. I like it because it paves the way for exactly the kind of conversation you’re mentioning: as media queries change and adapt over time, so too can this new element.
And the best part is that the HTML5 dictates that sources that don’t match the media attribute are never requested. If implemented correctly, it does exactly what you’ve said: grab the source we want, and bypass all the rest.
Some of our first iterations involved <noscript> tags and [removed]. I still have nightmares about dynamically-injected base tags. These were desperate times, and we left no stone unturned.
What problems did you find while using the <noscript> tags? I have spent quite some time with this technique, and so far it is the best method I have used to deliver proper images based on browser sizes.
I made a script for using the noscript tag technique without any pain. It is small and fast, but yet flexible. Please give it a spin:
“https://github.com/frederfred/respimg”:https://github.com/frederfred/respimg
The good part is that it involves no backend nor cookies and it works just perfect without JavaScript enabled.
Anyway, let’s hope that browsers will implement a clean solution for this any time soon.
Fantastic work on a tricky problem. I agree that JS should not handle img sizing and server side tricks are just not applicable, I can imagine many situations where server side programmers are gonna get well pissed with front end guys requesting, new and different solutions. However, I believe CSS to be the route that should be used, after all CSS is presentational whilst mark-up describes content and in this context I would regard imgs as presentational.
frederfred: Clever stuff, man! I’m not gonna lie: as frustrating as the responsive images problem has been at times, it has given way to some truly brilliant hacks. It ain’t always pretty (see also: some of our attempts at swapping <video> posters on-the-fly), but it leads to some wildly creative approaches.
Though, as Scott Jehl once said of a particularly desperate brainstorming session: “if this conversation gets leaked to the press, we’re both going to web standards hell.”
What about the <object> tag? I have to admit, I’ve never used <object> to post an image, and I’ve only had other programs generate its sloppy markup for me. But displaying images is one of its purposes, as listed in the spec. It also has a built-in and well-documented fallback.
Copy & paste the code below to embed this comment.
thezenmonkey
What about using a user agent query in the htaccess instead of the cookie. Basically redirect image requests for only mobile browsers. Now I know using User Agent Strings to detect something like an iPhone isn’t best way to do it, because User Agents can be spoofed but realistically aren’t those edge cases, like ie5 visits :)
Copy & paste the code below to embed this comment.
xorock
There is also another problem related with images. Lazy loading. With HTML5 we got async attribute to script element but no way to postpone image loading, force client to wait until image is really needed. For example imagine CSS3 based carousel with 10, 100 KB each sized images. We need to fallback to JS code to inject images to the page. In my opinion there should be something like test.png, when visible browser should request it and then apply all css rules (including effects, like transitions).
thezenmonkey: That’s possible, for certain—though, as you mention, we’d be opening ourselves up to various the issues that UA-detection entails. Spoofing, for one thing, and managing a growing and evolving list of devices for another. It becomes a bit more reasonable knowing we’d likely default to a smaller image and send the full-size image down the pipe only if a desktop UA string is matched (as we always want to fail in a way that’s still beneficial to users), but it still feels more like a stopgap solution to me.
Personally I love the approach being presented in the article. I have often found that many developers over-complicate solutions when something could be just as simple as a picture tag.
It would be nice to find a solution for responsive images that really works and is accepted cross browser.
Even if something like this were to be come standard, how long would it be till most browsers were implementing it?
Mat’s essay mentions that the work-around hacks by his team and others fails when ‘newer’ (and unmentioned) browsers may, upon parsing a link or via markup, proactively download things.
Work-arounds that rely on cookies and/or script-replacement may see issues, as the Boston Globe does now.
A semantic note: a pre-_fetch_ is, at present, a request to pre-load resources, on an individual basis. One HTML file, one image, etc. I do not believe any browsers parse said HTML and load its referenced resources when in this mode. A pre-_render_, on the other hand, loads a page and all its content, but keeps it hidden. At the moment, “it only happens on Chrome”:http://code.google.com/chrome/whitepapers/prerender.html . These are different from a DNS-pre-fetch, which simply resolves IP addresses for faster browser-to-server first-contact.
When Firefox pre-fetches content (at the behest of the referrer page’s markup), it sends the following header with the request: X-moz: prefetch
Safari does similarly, using: X-Purpose: preview. According to “this ticket”:http://code.google.com/p/chromium/issues/detail?id=58459 , Chrome does, too.
For pre-rendering, Chrome does not send any header whatsoever to the client. Instead, one must use the “Page Visibility API”:http://code.google.com/chrome/whitepapers/pagevisibility.html, in JS.
What to do
In most circumstances, a pre-fetch ought not to be initiated on a mobile browser without a lot of forethought. If you think said mobile browsers will hold to that idea (and I hope you’re right), you can watch for the HTTP pre-fetch headers, and make sure to load the full-size image for that request.
The various browsers need to be lobbied for an opt-out, too. While at the time, I thought the “effectively-mandated”:http://www.alistapart.com/articles/beyonddoctype IE-8 X-UA-Compatible header was horrendous, at least it put the capacity for control in the hands of an individual site. As browser-makers add whiz-bang features to their products, we as site builders need mechanisms to deliberately opt-out of those that would negatively affect user experience.
The DNS-prefetch implementors did this right. The X-DNS-Prefetch-Control header allows a site-owner to opt-out gracefully.
Remember the early days of the web, watching interlaced GIFs load, bit by bit, as though looking through a window whose blinds were being slowly opened? No? Get off my lawn.
Through a different methods, progressive image loading is supported by PNG and JPEG, as well. I don’t see anything regarding the new kid, WebP.
Imagine, for a moment, a single image file that would look correct on browsers of any size and connectivity. In theory, this is possible when interlaced images are handled properly.
A 800×800px image has four times as many pixels as its imaginary mobile-sized counterpart, which clocks in at 400×400px. Now imagine this larger image is a progressive image. In order to display this image well at the smaller, mobile size, one could theoretically load only one-quarter of the original image.
(Yes, I know there are overheads to progressive images. An image file, when saved progressively, is usually slightly larger than a non-progressive file. “There can be no progress, no achievement without sacrifice…“ -James Allen)
That sounds like a good future, doesn’t it? Because the browser understands the rendering intention (size), it can load only as much of the image as it needs to display it well. A really intelligent browser would consider this a partial download, and remember the file offset in case it wanted an even bigger image and decided to resume download.
The best of all worlds, without new markup.
Now if only someone would support JPEG2000, which is supposed to do an even better job at interlacing.
wessman: Oh dude, that is some serious knowledge you’ve just dropped. It’s very possible this means our original responsive images script is still viable. Granted it’s still a workaround for a pretty fundamental issue—and in no way changes my upcoming letter to Santa/the WHATWG—but this is absolute gold.
If you should ever find yourself in Boston, sir, I owe you a beer.
It seems to me that any technique that requires web developers to implement additional complex markup is only ever going to have a low takeup. If a device can only render a low res image it should probably make that clear in the headers of its request, and then web servers can scale images before sending them. This kind of approach can be implemented transparently and would work with existing sites and markup.
Detective work is not an easy thing to accomplish, but the image element replaced with an audio/video like approach is brilliant. Not only do we need to detect for resolution/screen size for imagery, but also scripts and libraries that are not needed for certain devices.
The Filament Group implemented EnhanceJS which is brilliant and totally useful for detection, but as of today there isn’t a ton of interest on their google code page. Also currently Modernizr also wont detect the max-device-width media query for its load property.
Too bad these options and solutions are not bringing in attraction as we seemed to be concerned with image sizes rather than the whole ball of wax. Thanks for the deep thoughts and share :)p
Hey Mat, really good article on image options for mobile first development. I was very curious on how we, as designers and developers, could get in on the conversations with WHATWG and the like about implementing better options for images in HTML markup.
Is there an easy way to join the conversation and would it be helpful to have more people in on it?
jkane001: We thought about that, actually—it turns out <image> can be used interchangeably with <img> in some older browsers.
Like I mention in “this comment”:http://www.alistapart.com/comments/responsive-images-how-they-almost-worked-and-what-we-need/P30/#31 , I’m not 100% in love with “picture” for the new element—but, y’know, it’s a theoretical work in progress.
Guys, thank you very much for all the feedback! This is the only way to get this topic in the right direction and to get a working code.
As a result I’ve spend some hours reading, analyzing the comments and sorted out. This is what is valuable feedback (sorry, I sorted out all the duplicates of code-element-names etc.): “See my updated article”:http://novolo.de/blog/2012/01/19/what-happened-to-responsive-assets/#update1
@Mat: What do you plan on the WHATWG list? I’d recommend moving the topic forward in W3C’s “public-html”-list where the topic is already in list here: “[whatwg] add html-attribute for responsive images”:http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2012-January/thread.html#34494
So please don’t open up a new duplicate while another topic has already run through the WHATWG lists and moved to W3C’s list. Thanks!
If you have questions, please don’t hesitate to contact me.
Copy & paste the code below to embed this comment.
my ad
I haven’t read all of the responses here but I’ve been thinking over this. I would love to see a ‘path’ option that could be included as part of the CSS and be a distinction in the media query. So for a normal 960 or larger view you could have img path:fullsize; and that could switch for tablets and mobile depending on the media query to be path:tablet; or path:mobile; as needed which would pull the appropriate image source from the folder that is being called. It is a bit of work easily solved through resizing actions. Much less work than what is currently needed with video and the various formats that are needed to degrade properly.
@my ad: As far as I understood you’re talking about a solution to solve the responsive image problem inside CSS, too. This is something you can read in my proposal here: “see blog article”:http://novolo.de/blog/2012/01/19/what-happened-to-responsive-assets/.
I like the idea of being responsive in either way — informational elements in HTML, non-informational/styling-elements in CSS.
Is that what you’ve been thinking of? If not, please clarify so we can sort this out.
Copy & paste the code below to embed this comment.
my ad
@anselmh Yes, I would like to see the solution come through CSS as I think it is the best way to approach it for the long term. Adding in javascript workarounds seems messy to me as we look into the future and think it only serves as a crutch to the bigger issues.
Thinking this through over the morning I’ve found various issues with my original thought and what could be solutions or could lead to complications. Below is a breakdown of how it could work changing around how a few things are currently structured.
Through the img tag you would nolonger specify an full path to the asset as it is currently structured. ie, img src=“folder/logo.jpg”
This would be handled through the CSS file using ‘path’ or some other specifier. So the document would now look as follows.
HTML:
img src=“logo.jpg”
CSS:
img { path: desktop; }
This would lead the logo.jpg to be pulled from an image folder named ‘desktop’ and retrieve the full size asset.
This can be changed using media queries to pull the different size assets from a different source folder as follows.
You could specify where the assets are coming from depending on the different scenarios that are being designed for. As we look ahead screen sizes and resolutions are going to keep multiplying and we’ll very soon be taking into account televisions adding more complexities.
Using this approach front end devs that are not concerned with responsive design can still use the traditional path structure without specifying path: ; in the CSS – img src=“folder/logo.jpg” and it would work as expected.
The part that I’m not fond about is having to create various sizes of the same image however this also opens up a number of design opportunities as you can customize images depending on which display it is being viewed on.
I see this providing designers a lot of potential to customize how a page would look and deliver images to that specific device instead of needing to use the same image for every instance and simply scaling it.
This approach keeps the current responsive principles and allows for additional customization but my favorite part of this solution is it only adds one piece of code and leaves the rest of the styled elements via CSS as is.
Copy & paste the code below to embed this comment.
my ad
I should clarify when I say ‘I would like to see the solution come through CSS’. Ideally the best solution would be native HTML enhanced by CSS. The HTML would be a fall back to an enhanced CSS solution.
I still think that that there should be an all emcompassing <media> tag that would allow the browser to gracefully degrade from <video> to <img> to any other element you could use for that area (and dependent on a standard media attribute). It seems like the video tag is only scratching the surface of the power of graceful degradation via HTML, why stop there? What about using a shorter headline or paragraph text if the viewport is smaller? Why not give us the option of giving a html alternative for an image versus relying on basic text in an alt tag?
Copy & paste the code below to embed this comment.
JoeSnellPDX
As our mobile-large screen responsive evolution presses on at warp speed… those on the crest of the wave are held back by seemingly simple technological hurdles. We have to remember the large screen in the equation as brand channels will begin to immerge with one feed -> mobile-large screen.
Our frustrations are but a small slice in time. What may seem like a solution today, may be an obstacle in only a matter of months.
I so love your suggested solution. It absolutely makes sense – today… and is evolution enabled.
Well said and good luck in persuading the powers that be to enable these markup improvements.
This is excellent article in the way it expresses what developers go through in battling how things change and evolve on the web and whereby most designers don’t ever really get the intricacies that developers have to deal with.
I think it’s not enough for designers to just study design, they need to study structure and code as well. The reverse is equally important but I think there’s plenty of momentum lately for developers to understand the importance of great design. Full disclosure, as a former developer, I may be biased in who I think needs to work harder. :) Nevertheless, nice article!
All these approaches put to much of a burden on developers! They’re clunky, prone to breakage and force the use of more markup. But maybe worst of all they force you to define fixed dimensions where you switch the image and to maintain multiple versions of the image!
I’m really quite astonished that no one so far has mentioned that JPEG 2000 together with the JPIP protocol handles this use case already… Store the image on the server in the maximum resolution you want to offer, but just download a size appropriate for the user’s display. Open source JPIP servers are available… (Djatoka for instance can serve ‘regular’ formats from a jp2 file, providing a fallback.) Better native support for JPEG 2000 in browsers would be nice of course.
Some experiments need to be done on how best to use inline JPEG 2000 for responsive images. I suppose one approach could be sending a scaling factor (or max size) with a cookie and do some URI rewriting on the server side to get the desired image size. CSS can still be used to downscale the image for a perfect fit. It’s not optimal, but the main point is to just work with one image and let the server handle the scaling.
Yes, JPEG 2000 requires encoders/decoders that are complex and more computationally ‘demanding’. But there has to be a tradeoff somewhere.
And let’s not forget that JPEG 2000 offers many other useful features.
I wrote the comment above in a hurry and forgot to clarify that of course no browser currently implements the JPIP protocol. However I still think using a JPEG 2000 server over HTTP could be a very interesting method towards solving the problem of responsive images. And it would certainly help push native implementations of the JPEG 2000 standard.
Copy & paste the code below to embed this comment.
JCOELHO
What about an approach of using the CSS media type to control the image source? No JavaScript involved, but it does assume that mobile devices come up with a media type of handheld. You probably can’t use the img tag, but maybe a div tag. Then again it changes the function of the div tag and is not semantically correct. A change to the markup to support images like videos would still be the best option.
I’ve been experimenting with this, . I watched the resources as I expanded the window and the different image for the widest display isn’t requested until that CSS is triggered by the window reaching each specified width. (I cused different images so I could see the change easily).
I agree, a picture tag was perfect to solve this problem, we have a solution for this, but this solution is not easy to implement, and, at this time, we have to care about many device sizes more and more. Hey W3C, look this post! :P
@Marc Diethelm: This is entirely the best solution. I’ve written about it as the “perfect solution” in my blog post. The problem is, that JPEG2000 isn’t that good of a format for graphics. So we would need ideally WebP/WebM to be able to progressively download and serve responsive images. Until that is implemented in all browsers (haha!) you will still need another approach, so that’s why we proposed the HTML5 solution for interim.
JCOELHO, yeosteve: A CSS only solution wouldn’t work as it’s not semantic and doesn’t work valid. It still should be possible to do via CSS for non-semantic (styling/layout) purposes, too. Mailing list of W3C didn’t care about this very much ;)
@Anserson Schmitt: We (Mat and I) are currently discussing on WHATWG lists and if we have a solution over there, it will move to the W3C list. So be sure we do our best W3C cares about this problem :)
@ NatCk: adaptive-images.com is only a polyfill solution and uses some technique not very good for a standard-solution.
Since we’re talking about changing the browser’s behavior anyway, why not actually make it offer up the information to properly solve this problem server-side? Let’s have the browser send the information that is available to the “css media queries”:http://www.w3.org/TR/css3-mediaqueries/ in the HTTPGET request headers caused by the one-and-only whatever.png tag or CSS url(“whatever.png”).
To wit:
X-Media: “screen color:24 resolution:120dpi orientation:portrait device:640×480 viewport:500×300”
(obviously, the aspect-ratio and device-aspect-ratio can be computed, color-index seems meh)
The nice thing about this is that it is really up to the user-agent’s current needs to determine what should be displayed, thus what should be requested. If someone zooms in, the user-agent can ask for a bigger version. If they switch to print view, we can ask for the monochrome version in higher DPI and all is warm and fuzzy. If the server doesn’t care, it serves up the one-true-image. If the server cares, it can serve specific versions. Cache logic can be driven by the standard content-negotiation logic (and left non-cached until it is).
The best part of ALL of this is that the CSS can just reference an image, the html can just include a img tag, etc… and the browser’s request can be driven by what it knows already about the size/placement/device of the image.
Use JavaScript stylesheet manipulation. All responsive images should be rendered in an <img> with a token classname, such as ‘responsive’, and should have fallback ‘src’ value. In <head>, JS will add a stylesheet rule to ‘display:none’ all such elements (to avoid requesting the wrong image resource), then will determine the appropriate screen size category. Script block at the bottom of the page will alter ‘src’ values of all responsive <img>s according to screen size category, per some naming convention, and then update stylesheet rule to ‘display:inline’ all such elements.
This may cause flicker, more noticeable when HTML is generated/delivered slowly. This can be mitigated by encapsulating the responsive <img> in some server-side component that also emits a script block right after each <img> to do the ‘src’ attribute manipulation and displaying.
The ‘resize’ event will be handled to perform this logic in case different images need to be re-rendered (can be short-circuited if new image would have lower quality).
If UA doesn’t support JS or doesn’t support stylesheet manipulation, there will be flicker or simply the fallback content is rendered.
This one is an inspiration personally to uncover out rather more related to this subject. I must confess your information extended my sentiments as well as I am going to proper now take your feed to stay updated on every coming blog posts you would possibly possibly create. You are worthy of thanks for a job perfectly carried out!
The “Responsive Content”:http://stephanfowler.github.com/responsive-content/ jQuery plugin is an approach which works in a “coarse grained” way, ajax-loading an entire HTML fragment into the page, given current window width (and pixel density). It can be used to tell the server to supply appropriately sized image src URLs etc.
It’s a fork of Github’s Pjax loader. It does not rely on User Agent or cookies – just window width.
Copy & paste the code below to embed this comment.
DanMcRae
How do would you appreciate these 2 solutions to download only the small images when needed (Smartphone, iPhone, Android)
1) Use $_SERVER(‘HTTP_USER_AGENT’) in the server script to detect that it is not a PC
2) Have a main page that detects the screen size and pass it in the URI for subsequent pages.
75 Reader Comments
Back to the ArticleScott Kellum
A long, long time ago the “lowsrc”:http://lists.w3.org/Archives/Public/www-html/2002Nov/0136.html attribute would handle something very similar to this. It would be nice to have it back again to quickly load an image on any browser, then once everything is in place start working on high res if possible. A non-js implementation would be ideal, as you wrote towards the end of your post better fallbacks and conditional loading would be wonderful. Some other options for responsive loading were discussed on “quirksmode”:http://www.quirksmode.org/blog/archives/2011/09/responsive_desi.html ,
John Bertucci
I think this idea of a element.
One thing that caught my eye was the repeated idea of:
<source src=“mobile.jpg”>
mobile.jpg
Wouldn’t it be nicer to simply use [img]big-img.jpga4SRO0xrVj8VBqz7kAMQ96V0hvUt5xsBtypography_img_src_end media=“min-width:800px”> as the standard image so you only need to declare it once and add alternative images?
Ie:
<source [/img]
mobile.jpg
</picture>
spaceribs
This design pattern should be broken out for all media really:
<media>
<video src=“high-res.webm” media=“min-width:800px” />
<video src=“low-res.webm” />
poster.jpg
poster.jpg
</media>
patrickarlt
[img]medium.pnga4SRO0xrVj8VBqz7kAMQ96V0hvUt5xsBtypography_img_src_end media=“min-width:600px;”> has been a closed tag forever, why not just allow the tag to be opened and use a format like.
fallback.png
<source [/img]
<source src=“full.png” media=“min-width:960px;”>
</img>
No new tag, older content still works, and we get real responsive images.
natewalton
I’ve looked at it in a similar way to @steveu in comment 17. I’ve experimented with loading the small image for all browsers (sized up if necessary) and replacing it with the larger image via javascript as needed. As you mention, in high-bandwidth situations users get a flash of the blurry image, which resolves very quickly to the high-res image if the bandwidth is decent. It’s maybe not the most ideal (I like the new markup proposal) but it works adequately in every case.
In my mind, this approach has the added advantage of a super-fast initial display for everyone (including desktops), since they only have to download the teeny images at first. From what I’ve seen, many people will be okay with a progressive image load if it makes the site faster.
JohnAlbin_
The <img> tag can’t be opened up because of the way legacy browsers treat it as a self-closing tag.
<img>
more stuff
</img>
The </img> tag would be treated as another img element in the DOM. And none of the contents between the <img> and the </img> would be considered as child elements. And a </img> with no src would hurt performance. See http://www.nczonline.net/blog/2009/11/30/empty-image-src-can-destroy-your-site/
You have to have a new tag.
Neal G
Why not use Data URI or even just a blank image and then use Javascript to decide what image to use?
data:image/gif,…
yeosteve
It is becoming clear to me that “responsive” design leaves all layout choices in the hands of the designers (where it should be) and “adaptive” requires more consultation with the developers, and the best result will probably be a combination of them. I would like to see the web change to offering as much control over layout and content to the front end developers as possible, but we’re not there yet.
For a simple page that only contains content you want to display anywhere, if the images can be delivered as CSS background images then surely @media queries would be way to go, specifying a different image depending on screen size.
If the design has been created in that quaint, print-focussed, Photoshop-a desktop-design-first approach, then a mobile site may have to be assembled with a controller selecting a different set of models and views depending on the user agent.
Right now I’m advocating adaptive techniques as a fall back, and they are often the only way to effectively “mobilize” an old-fashioned web site reasonably quickly. The benefit of this is that the client will soon want their desktop site to be as efficient and user-friendly as the mobile version.
Keep the pressure
Captain Genius
Certainly the news is only grim for client-side approaches. For those of us with CMSes or access to php, the outlook has to be more positive.
JohnAlbin_
I’d strongly encourage everyone to read “Jason Grigg’s article on responsive images”:http://www.cloudfour.com/responsive-imgs/ (which Mat linked to.)
It explains a lot of the difficulties that you have to work around in order to develop a responsive image solution. By far the worst one is the “First page load problem”. If you want to use JavaScript to determine the capabilities of the user’s browser, you can’t do that before you send your first page’s content. And you can’t try to use some blocking JS in the <head> that determines capabilities before the body is parsed”¦ because of the new techniques Mat mentioned where they pre-fetch images before building up the DOM of the body. And you can’t use data-URIs and “blank images” because “a <img> with no src is not actually blank”:http://www.nczonline.net/blog/2009/11/30/empty-image-src-can-destroy-your-site/ .
Which means on the first connection, you don’t know anything about the browser, so you can’t send “the right image” to it. If you want to try to figure out some of the capabilities of the browser before you send the first page of data, you have to rely on UA sniffing. UA sniffing is fragile and requires constant updating of your UA database.
So, yes, you can build server-side solutions into a CMS, but that UA-sniffing code will never be as robust as a native HTML solution could be. Personally, I’d like to try to build a server-side solution that uses a Future Friendly markup with the element.
Mat Marquis
DavidSparks: Yeah, I’m not 100% married to “picture,” to represent the new element. “Poster” would make more sense, but it’s a term already in use and that might trip people up. Maybe “art?” Stuff we can nail down once we have something working, I figure.
In terms of the media attributes, you could still specify media=“screen” and the like in there—you can use any combination of media queries, the same as you would in your CSS. I like it because it paves the way for exactly the kind of conversation you’re mentioning: as media queries change and adapt over time, so too can this new element.
And the best part is that the HTML5 dictates that sources that don’t match the media attribute are never requested. If implemented correctly, it does exactly what you’ve said: grab the source we want, and bypass all the rest.
chad
I really like it. The time is right.
frederfred
What problems did you find while using the <noscript> tags? I have spent quite some time with this technique, and so far it is the best method I have used to deliver proper images based on browser sizes.
I made a script for using the noscript tag technique without any pain. It is small and fast, but yet flexible. Please give it a spin:
“https://github.com/frederfred/respimg”:https://github.com/frederfred/respimg
The good part is that it involves no backend nor cookies and it works just perfect without JavaScript enabled.
Anyway, let’s hope that browsers will implement a clean solution for this any time soon.
Surrey Web
Fantastic work on a tricky problem. I agree that JS should not handle img sizing and server side tricks are just not applicable, I can imagine many situations where server side programmers are gonna get well pissed with front end guys requesting, new and different solutions. However, I believe CSS to be the route that should be used, after all CSS is presentational whilst mark-up describes content and in this context I would regard imgs as presentational.
Mat Marquis
frederfred: Clever stuff, man! I’m not gonna lie: as frustrating as the responsive images problem has been at times, it has given way to some truly brilliant hacks. It ain’t always pretty (see also: some of our attempts at swapping <video> posters on-the-fly), but it leads to some wildly creative approaches.
Though, as Scott Jehl once said of a particularly desperate brainstorming session: “if this conversation gets leaked to the press, we’re both going to web standards hell.”
ragdoll
What about the <object> tag? I have to admit, I’ve never used <object> to post an image, and I’ve only had other programs generate its sloppy markup for me. But displaying images is one of its purposes, as listed in the spec. It also has a built-in and well-documented fallback.
I wonder if something like that could work.
thezenmonkey
What about using a user agent query in the htaccess instead of the cookie. Basically redirect image requests for only mobile browsers. Now I know using User Agent Strings to detect something like an iPhone isn’t best way to do it, because User Agents can be spoofed but realistically aren’t those edge cases, like ie5 visits :)
xorock
There is also another problem related with images. Lazy loading. With HTML5 we got async attribute to script element but no way to postpone image loading, force client to wait until image is really needed. For example imagine CSS3 based carousel with 10, 100 KB each sized images. We need to fallback to JS code to inject images to the page. In my opinion there should be something like test.png, when visible browser should request it and then apply all css rules (including effects, like transitions).
Mat Marquis
thezenmonkey: That’s possible, for certain—though, as you mention, we’d be opening ourselves up to various the issues that UA-detection entails. Spoofing, for one thing, and managing a growing and evolving list of devices for another. It becomes a bit more reasonable knowing we’d likely default to a smaller image and send the full-size image down the pipe only if a desktop UA string is matched (as we always want to fail in a way that’s still beneficial to users), but it still feels more like a stopgap solution to me.
plumwd
Personally I love the approach being presented in the article. I have often found that many developers over-complicate solutions when something could be just as simple as a picture tag.
It would be nice to find a solution for responsive images that really works and is accepted cross browser.
Even if something like this were to be come standard, how long would it be till most browsers were implementing it?
wessman
Mat’s essay mentions that the work-around hacks by his team and others fails when ‘newer’ (and unmentioned) browsers may, upon parsing a link or via markup, proactively download things.
Work-arounds that rely on cookies and/or script-replacement may see issues, as the Boston Globe does now.
A semantic note: a pre-_fetch_ is, at present, a request to pre-load resources, on an individual basis. One HTML file, one image, etc. I do not believe any browsers parse said HTML and load its referenced resources when in this mode. A pre-_render_, on the other hand, loads a page and all its content, but keeps it hidden. At the moment, “it only happens on Chrome”:http://code.google.com/chrome/whitepapers/prerender.html . These are different from a DNS-pre-fetch, which simply resolves IP addresses for faster browser-to-server first-contact.
When Firefox pre-fetches content (at the behest of the referrer page’s markup), it sends the following header with the request:
X-moz: prefetchSafari does similarly, using:
X-Purpose: preview. According to “this ticket”:http://code.google.com/p/chromium/issues/detail?id=58459 , Chrome does, too.For pre-rendering, Chrome does not send any header whatsoever to the client. Instead, one must use the “Page Visibility API”:http://code.google.com/chrome/whitepapers/pagevisibility.html, in JS.
What to do
In most circumstances, a pre-fetch ought not to be initiated on a mobile browser without a lot of forethought. If you think said mobile browsers will hold to that idea (and I hope you’re right), you can watch for the HTTP pre-fetch headers, and make sure to load the full-size image for that request.
The various browsers need to be lobbied for an opt-out, too. While at the time, I thought the “effectively-mandated”:http://www.alistapart.com/articles/beyonddoctype IE-8
X-UA-Compatibleheader was horrendous, at least it put the capacity for control in the hands of an individual site. As browser-makers add whiz-bang features to their products, we as site builders need mechanisms to deliberately opt-out of those that would negatively affect user experience.The DNS-prefetch implementors did this right. The
X-DNS-Prefetch-Controlheader allows a site-owner to opt-out gracefully.wessman
Remember the early days of the web, watching interlaced GIFs load, bit by bit, as though looking through a window whose blinds were being slowly opened? No? Get off my lawn.
Through a different methods, progressive image loading is supported by PNG and JPEG, as well. I don’t see anything regarding the new kid, WebP.
Imagine, for a moment, a single image file that would look correct on browsers of any size and connectivity. In theory, this is possible when interlaced images are handled properly.
A 800×800px image has four times as many pixels as its imaginary mobile-sized counterpart, which clocks in at 400×400px. Now imagine this larger image is a progressive image. In order to display this image well at the smaller, mobile size, one could theoretically load only one-quarter of the original image.
(Yes, I know there are overheads to progressive images. An image file, when saved progressively, is usually slightly larger than a non-progressive file. “There can be no progress, no achievement without sacrifice…“ -James Allen)
That sounds like a good future, doesn’t it? Because the browser understands the rendering intention (size), it can load only as much of the image as it needs to display it well. A really intelligent browser would consider this a partial download, and remember the file offset in case it wanted an even bigger image and decided to resume download.
The best of all worlds, without new markup.
Now if only someone would support JPEG2000, which is supposed to do an even better job at interlacing.
Mat Marquis
wessman: Oh dude, that is some serious knowledge you’ve just dropped. It’s very possible this means our original responsive images script is still viable. Granted it’s still a workaround for a pretty fundamental issue—and in no way changes my upcoming letter to Santa/the WHATWG—but this is absolute gold.
If you should ever find yourself in Boston, sir, I owe you a beer.
James Gaunt
It seems to me that any technique that requires web developers to implement additional complex markup is only ever going to have a low takeup. If a device can only render a low res image it should probably make that clear in the headers of its request, and then web servers can scale images before sending them. This kind of approach can be implemented transparently and would work with existing sites and markup.
grayghostvisuals
Detective work is not an easy thing to accomplish, but the image element replaced with an audio/video like approach is brilliant. Not only do we need to detect for resolution/screen size for imagery, but also scripts and libraries that are not needed for certain devices.
The Filament Group implemented EnhanceJS which is brilliant and totally useful for detection, but as of today there isn’t a ton of interest on their google code page. Also currently Modernizr also wont detect the max-device-width media query for its load property.
Too bad these options and solutions are not bringing in attraction as we seemed to be concerned with image sizes rather than the whole ball of wax. Thanks for the deep thoughts and share :)p
jkane001
In keeping with new media tags, why not just “<image>”?
jasonmerino
Hey Mat, really good article on image options for mobile first development. I was very curious on how we, as designers and developers, could get in on the conversations with WHATWG and the like about implementing better options for images in HTML markup.
Is there an easy way to join the conversation and would it be helpful to have more people in on it?
Thanks!
Mat Marquis
jkane001: We thought about that, actually—it turns out <image> can be used interchangeably with <img> in some older browsers.
Like I mention in “this comment”:http://www.alistapart.com/comments/responsive-images-how-they-almost-worked-and-what-we-need/P30/#31 , I’m not 100% in love with “picture” for the new element—but, y’know, it’s a theoretical work in progress.
anselmh
Guys, thank you very much for all the feedback! This is the only way to get this topic in the right direction and to get a working code.
As a result I’ve spend some hours reading, analyzing the comments and sorted out. This is what is valuable feedback (sorry, I sorted out all the duplicates of code-element-names etc.): “See my updated article”:http://novolo.de/blog/2012/01/19/what-happened-to-responsive-assets/#update1
@Mat: What do you plan on the WHATWG list? I’d recommend moving the topic forward in W3C’s “public-html”-list where the topic is already in list here: “[whatwg] add html-attribute for responsive images”:http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2012-January/thread.html#34494
So please don’t open up a new duplicate while another topic has already run through the WHATWG lists and moved to W3C’s list. Thanks!
If you have questions, please don’t hesitate to contact me.
Cheers,
Anselm
my ad
I haven’t read all of the responses here but I’ve been thinking over this. I would love to see a ‘path’ option that could be included as part of the CSS and be a distinction in the media query. So for a normal 960 or larger view you could have img path:fullsize; and that could switch for tablets and mobile depending on the media query to be path:tablet; or path:mobile; as needed which would pull the appropriate image source from the folder that is being called. It is a bit of work easily solved through resizing actions. Much less work than what is currently needed with video and the various formats that are needed to degrade properly.
anselmh
@my ad: As far as I understood you’re talking about a solution to solve the responsive image problem inside CSS, too. This is something you can read in my proposal here: “see blog article”:http://novolo.de/blog/2012/01/19/what-happened-to-responsive-assets/.
I like the idea of being responsive in either way — informational elements in HTML, non-informational/styling-elements in CSS.
Is that what you’ve been thinking of? If not, please clarify so we can sort this out.
my ad
@anselmh Yes, I would like to see the solution come through CSS as I think it is the best way to approach it for the long term. Adding in javascript workarounds seems messy to me as we look into the future and think it only serves as a crutch to the bigger issues.
Thinking this through over the morning I’ve found various issues with my original thought and what could be solutions or could lead to complications. Below is a breakdown of how it could work changing around how a few things are currently structured.
Through the img tag you would nolonger specify an full path to the asset as it is currently structured. ie, img src=“folder/logo.jpg”
This would be handled through the CSS file using ‘path’ or some other specifier. So the document would now look as follows.
HTML:
img src=“logo.jpg”
CSS:
img { path: desktop; }
This would lead the logo.jpg to be pulled from an image folder named ‘desktop’ and retrieve the full size asset.
This can be changed using media queries to pull the different size assets from a different source folder as follows.
@media screen and (max-width: 480px) {
img { path: mobile; }
}
If a person doesn’t like to have all images coming out of one directory you could pull images out of various folders through a class element.
img.company { path: company; }
img.products { path: products; }
resulting in – img class“company” src=“logo.jpg”
and so on.
You could specify where the assets are coming from depending on the different scenarios that are being designed for. As we look ahead screen sizes and resolutions are going to keep multiplying and we’ll very soon be taking into account televisions adding more complexities.
Using this approach front end devs that are not concerned with responsive design can still use the traditional path structure without specifying path: ; in the CSS – img src=“folder/logo.jpg” and it would work as expected.
The part that I’m not fond about is having to create various sizes of the same image however this also opens up a number of design opportunities as you can customize images depending on which display it is being viewed on.
I see this providing designers a lot of potential to customize how a page would look and deliver images to that specific device instead of needing to use the same image for every instance and simply scaling it.
This approach keeps the current responsive principles and allows for additional customization but my favorite part of this solution is it only adds one piece of code and leaves the rest of the styled elements via CSS as is.
my ad
I should clarify when I say ‘I would like to see the solution come through CSS’. Ideally the best solution would be native HTML enhanced by CSS. The HTML would be a fall back to an enhanced CSS solution.
spaceribs
I still think that that there should be an all emcompassing <media> tag that would allow the browser to gracefully degrade from <video> to <img> to any other element you could use for that area (and dependent on a standard media attribute). It seems like the video tag is only scratching the surface of the power of graceful degradation via HTML, why stop there? What about using a shorter headline or paragraph text if the viewport is smaller? Why not give us the option of giving a html alternative for an image versus relying on basic text in an alt tag?
Too many tags, not enough scalability.
JoeSnellPDX
As our mobile-large screen responsive evolution presses on at warp speed… those on the crest of the wave are held back by seemingly simple technological hurdles. We have to remember the large screen in the equation as brand channels will begin to immerge with one feed -> mobile-large screen.
Our frustrations are but a small slice in time. What may seem like a solution today, may be an obstacle in only a matter of months.
I so love your suggested solution. It absolutely makes sense – today… and is evolution enabled.
Well said and good luck in persuading the powers that be to enable these markup improvements.
Responsive image nirvana.
NatCk
Commenter #11 asked about this but I didn’t see a response from Mat. Any thoughts on this approach?
http://adaptive-images.com/
eglue
This is excellent article in the way it expresses what developers go through in battling how things change and evolve on the web and whereby most designers don’t ever really get the intricacies that developers have to deal with.
I think it’s not enough for designers to just study design, they need to study structure and code as well. The reverse is equally important but I think there’s plenty of momentum lately for developers to understand the importance of great design. Full disclosure, as a former developer, I may be biased in who I think needs to work harder. :) Nevertheless, nice article!
timtowles
I’d like to propose a new tag: IMG
http://1997.webhistory.org/www.lists/www-talk.1993q1/0182.html
heh.
joyblack
I think it’s a good method. Not all people can think of ideas as great as the ones mentioned above.
http://fantasycoinhq.3dcartstores.com
joyblack
fantasy, coins, gold, silver, copper, D&D, role, playing, currency, money, campaign
Marc Diethelm
All these approaches put to much of a burden on developers! They’re clunky, prone to breakage and force the use of more markup. But maybe worst of all they force you to define fixed dimensions where you switch the image and to maintain multiple versions of the image!
I’m really quite astonished that no one so far has mentioned that JPEG 2000 together with the JPIP protocol handles this use case already… Store the image on the server in the maximum resolution you want to offer, but just download a size appropriate for the user’s display. Open source JPIP servers are available… (Djatoka for instance can serve ‘regular’ formats from a jp2 file, providing a fallback.) Better native support for JPEG 2000 in browsers would be nice of course.
Some experiments need to be done on how best to use inline JPEG 2000 for responsive images. I suppose one approach could be sending a scaling factor (or max size) with a cookie and do some URI rewriting on the server side to get the desired image size. CSS can still be used to downscale the image for a perfect fit. It’s not optimal, but the main point is to just work with one image and let the server handle the scaling.
Yes, JPEG 2000 requires encoders/decoders that are complex and more computationally ‘demanding’. But there has to be a tradeoff somewhere.
And let’s not forget that JPEG 2000 offers many other useful features.
Marc Diethelm
I wrote the comment above in a hurry and forgot to clarify that of course no browser currently implements the JPIP protocol. However I still think using a JPEG 2000 server over HTTP could be a very interesting method towards solving the problem of responsive images. And it would certainly help push native implementations of the JPEG 2000 standard.
JCOELHO
What about an approach of using the CSS media type to control the image source? No JavaScript involved, but it does assume that mobile devices come up with a media type of handheld. You probably can’t use the img tag, but maybe a div tag. Then again it changes the function of the div tag and is not semantically correct. A change to the markup to support images like videos would still be the best option.
yeosteve
I’ve been experimenting with this, . I watched the resources as I expanded the window and the different image for the widest display isn’t requested until that CSS is triggered by the window reaching each specified width. (I cused different images so I could see the change easily).
No Javascript at all.
.rightBox{
display:none;
width:40%;
background-size:contain;
background-repeat:no-repeat;
float:right;
overflow:auto
}
@media screen and (orientation:landscape) and (min-width:480px){
.rightBox{
display:block;
height:40%;
background-image:url(’../images/mobile/sevenstep.jpg’);
}
}
@media screen and (orientation:landscape) and (min-width:1024px){
.rightBox{
display:block;
height:40%;
background-image:url(’../images/full/planning.jpg’);
}
}
Anderson Schmitt
I agree, a picture tag was perfect to solve this problem, we have a solution for this, but this solution is not easy to implement, and, at this time, we have to care about many device sizes more and more. Hey W3C, look this post! :P
anselmh
@Marc Diethelm: This is entirely the best solution. I’ve written about it as the “perfect solution” in my blog post. The problem is, that JPEG2000 isn’t that good of a format for graphics. So we would need ideally WebP/WebM to be able to progressively download and serve responsive images. Until that is implemented in all browsers (haha!) you will still need another approach, so that’s why we proposed the HTML5 solution for interim.
JCOELHO,yeosteve: A CSS only solution wouldn’t work as it’s not semantic and doesn’t work valid. It still should be possible to do via CSS for non-semantic (styling/layout) purposes, too. Mailing list of W3C didn’t care about this very much ;)@Anserson Schmitt: We (Mat and I) are currently discussing on WHATWG lists and if we have a solution over there, it will move to the W3C list. So be sure we do our best W3C cares about this problem :)
@ NatCk: adaptive-images.com is only a polyfill solution and uses some technique not very good for a standard-solution.
IDisposable
Since we’re talking about changing the browser’s behavior anyway, why not actually make it offer up the information to properly solve this problem server-side? Let’s have the browser send the information that is available to the “css media queries”:http://www.w3.org/TR/css3-mediaqueries/ in the HTTP GET request headers caused by the one-and-only whatever.png tag or CSS url(“whatever.png”).
To wit:
X-Media: “screen color:24 resolution:120dpi orientation:portrait device:640×480 viewport:500×300”
(obviously, the aspect-ratio and device-aspect-ratio can be computed, color-index seems meh)
The nice thing about this is that it is really up to the user-agent’s current needs to determine what should be displayed, thus what should be requested. If someone zooms in, the user-agent can ask for a bigger version. If they switch to print view, we can ask for the monochrome version in higher DPI and all is warm and fuzzy. If the server doesn’t care, it serves up the one-true-image. If the server cares, it can serve specific versions. Cache logic can be driven by the standard content-negotiation logic (and left non-cached until it is).
The best part of ALL of this is that the CSS can just reference an image, the html can just include a img tag, etc… and the browser’s request can be driven by what it knows already about the size/placement/device of the image.
lunalot
Use JavaScript stylesheet manipulation. All responsive images should be rendered in an <img> with a token classname, such as ‘responsive’, and should have fallback ‘src’ value. In <head>, JS will add a stylesheet rule to ‘display:none’ all such elements (to avoid requesting the wrong image resource), then will determine the appropriate screen size category. Script block at the bottom of the page will alter ‘src’ values of all responsive <img>s according to screen size category, per some naming convention, and then update stylesheet rule to ‘display:inline’ all such elements.
This may cause flicker, more noticeable when HTML is generated/delivered slowly. This can be mitigated by encapsulating the responsive <img> in some server-side component that also emits a script block right after each <img> to do the ‘src’ attribute manipulation and displaying.
The ‘resize’ event will be handled to perform this logic in case different images need to be re-rendered (can be short-circuited if new image would have lower quality).
If UA doesn’t support JS or doesn’t support stylesheet manipulation, there will be flicker or simply the fallback content is rendered.
katerpiddah
Is there a reason why you can’t just use images as the source in the video tag with the image MIME type, as in the following example??
<video>
<source src=“high-res.jpg” media=“min-width:800px” type=“image/jpeg”>
<source src=“low-res.jpg” type=“image/jpeg”>
poster.jpg
</video>
Thanks.
sminfosoft10
This one is an inspiration personally to uncover out rather more related to this subject. I must confess your information extended my sentiments as well as I am going to proper now take your feed to stay updated on every coming blog posts you would possibly possibly create. You are worthy of thanks for a job perfectly carried out!
“Web Design Company”:http://www.sminfosoft.com
“Technology News”:http://update-technologynews.blogspot.in/
lerouxb
What do you think of my responsive images proposal?
http://opensores.za.net/2012/responsive-images/
stephanfowler
The “Responsive Content”:http://stephanfowler.github.com/responsive-content/ jQuery plugin is an approach which works in a “coarse grained” way, ajax-loading an entire HTML fragment into the page, given current window width (and pixel density). It can be used to tell the server to supply appropriately sized image src URLs etc.
It’s a fork of Github’s Pjax loader. It does not rely on User Agent or cookies – just window width.
stephanfowler
Correstion. Correct url:
http://responsivecontent.net/
Jessica Wilson
But, Do I Need Responsive Website For Growth of My Business?
DanMcRae
How do would you appreciate these 2 solutions to download only the small images when needed (Smartphone, iPhone, Android)
1) Use $_SERVER(‘HTTP_USER_AGENT’) in the server script to detect that it is not a PC
2) Have a main page that detects the screen size and pass it in the URI for subsequent pages.