A Q&A on the Picture Element

The revival of the picture element—the responsive images proposal that has seen the most support from the developer community—is exciting news, but there are still some outstanding questions about how the element will really work. Marcos Caceres and Yoav Weiss have put countless hours into the Responsive Images Community Group’s efforts, and are now working toward picture implementations in Firefox and Chrome, respectively. Mat Marquis asked them some questions.

So, we’re getting picture and srcset? I thought srcset was bad?

MC: srcset was never bad in itself—some parts of the syntax were just hard to understand, and it wasn’t able to handle an important use case: “art direction”. The picture element works in conjunction with srcset, giving developers a set of solutions for whatever problem they are trying to solve.

What happened to the src-n proposal that was going around a short time ago?

MC: The src-n proposal (put together by Google’s Tab Atkins and John Mellor) elegantly solved a lot of problems, but introduced some weird markup (the numbered attributes bit), which would have made a mess of browsers’ internals. Some WebKit folks went so far as to call it “a grotesque perversion of the HTML language.”

YW: The biggest innovation in src-n was the sizes attribute. This attribute allows you to specify the dimensions for a set of images and lets the browser take care of the math behind all the resource selection. We’ve incorporated that feature into the latest picture proposal—the src-n proposal was an important step in getting the complete solution that we have today.

I thought picture was done-for—what brought it back?

MC: It was really the rejection of src-n by WebKit that brought it back. By taking picture off the standardization table, there was a new sense of urgency to finding a solution for responsive images.

Mozilla was quite keen on src-n, as we thought that, despite being hard to implement, it did a fair job of addressing the problems developers were facing. But, when the WebKit community said no to src-n, the Blink community backed out as well—the Blink folks weren’t sold on it to start with, so they weren’t apt to do something the WebKit folks were not keen on either. With Mozilla having rejected the original srcset proposal, this really only left picture on the table.

Then, a clever dude from Opera software—Simon Pieters—had an epiphany: what if we flip the way that browsers process picture? If we used the picture element as sort of a controller for an old-fashioned img element, we could get all the same functionality with way less implementation overhead.

YW: The old proposal’s version of picture acted like a better featured version of img itself, in the new proposal the picture element is there only to contain possible resources for the img element, and assists it in choosing the right one.

img can keep doing what it does best: loading and displaying a resource, and the new element just handles the parts that img doesn’t excel at—namely, picking the most appropriate resource based on a combination of factors: viewport size, pixel density, and so on.

This design enables browsers to avoid re-implementation and re-testing of img’s core functionality with picture, and reduces the maintenance costs of the feature significantly.

Does picture need to land before we can do any real performance testing, or are there tests in place before the element officially hits browsers?

MC: I think picture will need to land before we can get any real numbers. However, given that srcset is already in Chrome 34, sites might already be seeing some of the benefits that come from a responsive image solution.

YW: Some of the performance benefits can be measured by using today’s polyfills. For example, the data savings from using picture are not likely to change significantly compared to the data savings benefits of using Picturefill, minus the actual polyfill download. One difference is that current polyfills have to work around browser-level optimizations—like prefetching sources—while a native solution will be able to take advantage of them.

How does the current implementation in Chrome differ from implementations in existing polyfills? Do I need to change my code to get it working natively or will it integrate seamlessly?

YW: The Blink implementation is of srcset’s DPR syntax, which is a subset of the original srcset syntax. If the polyfill you use has implemented that syntax (the x variant of srcset), and does feature testing, it’s highly possible that you won’t have to modify your site’s syntax.

MC: As with any emerging standard, things can change quickly. There is always risk in prematurely adopting a solution before it becomes a standard. Fortunately, the high visibility and level of interest in this feature means the community is already updating their polyfills. For example, Scott Jehl is planning to update Picturefill to support the new syntax and attributes of picture soon.

Will the type attribute on source be supported when picture is implemented in Chrome and Firefox?

MC: I don’t see any reason why it wouldn’t be. An important part of a responsive images solution is having the ability to use emerging image formats like WebP, particularly if using these formats will benefit users without excluding those still using legacy browsers. As a community, we need to make sure there is a solid test suite to check for support of the type attribute—and we need to hold them accountable if the browsers don’t get it right!

YW: What Marcos said!

How interoperable will picture be with MQ variables?

MC: In most cases, as long as you can use the final MQ variable syntax anywhere you would use a media query normally, then it should“just work”.

YW: The latest MQ variable proposal—dubbed “CSS aliases”—is still in its very early stages, but we’ve been thinking about how it might work with picture already. Interoperability with picture is going to be important for any MQ variable proposal.

Is bandwidth a consideration right now, or is this all mostly about viewport sizes and densities?

MC: Bandwidth detection itself isn’t a relevant factor. Consider, when you go to a conference, the Wi-Fi you connect to has high bandwidth, yet you get slow speeds. Bandwidth is variable—particularly on cellular connections. Most of the time,what users care about is costs, particularly on mobile. I think what we want is for users to have the ability to tell the browser, “If I’m on my cellular plan, just give me the 1x images.”

This is the beauty of the declarative picture solution: all developers have to do is provide a suitable set of images for the browser to choose from. It’s then on browsers to do the right thing, on behalf of the user.

You can expect that minimizing the cost of browsing is something we browser vendors will be competing on—a definite win for our users.

YW: Initial implementations likely won’t take network quality into account—but as Marcos said, the solution’s markup provides all the information the browser needs to later take that into account. All in all, the solution aims to give this control to the browsers, and through them, to the users themselves.

Is it safe to use the picture element in a production site, once these implementations launch?

MC: I would wait a little bit—until there’s public confirmation from all the big browser vendors—before using picture on a production site. I’d hate to see people include this prematurely, and a change to the markup come along later. With that caveat: the really nice thing about the new picture markup is that developers must include an img element for it to work. This means that by default—and with no exception—the fallback for picture will work with legacy browsers. It’s then up to authors to include an optimized image for legacy browsers inside the img element.

YW: In general I agree with Marcos, but I guess it also depends on the polyfills available and their support for the new markup. Once Picturefill and other polyfills support the markup, it might be possible to roll the new markup out to live sites that we control—like our own—as long as we make sure that we can adapt quickly in the unlikely event that the markup changes.