We all want our websites to be fast. We optimize images, create CSS sprites, use CDNs, cache aggressively, and gzip and minimize static content. We use every trick in the book.
But we can still do more. If we want faster outcomes, we have to think differently. What if, instead of leaving our users to stare at a spinning wheel, waiting for content to be delivered, we could predict where they wanted to go next? What if we could have that content ready for them before they even ask for it?
We tend to see the web as a reactive model, where every action causes a reaction. Users click, then we take them to a new page. They click again, and we open another page. But we can do better. We can be proactive with prebrowsing.
The three big techniques#section2
Steve Souders coined the term prebrowsing (from predictive browsing) in one of his articles late last year. Prebrowsing is all about anticipating where users want to go and preparing the content ahead of time. It’s a big step toward a faster and less visible internet.
Browsers can analyze patterns to predict where users are going to go next, and start DNS resolution and TCP handshakes as soon as users hover over links. But to get the most out of these improvements, we can enable prebrowsing on our web pages, with three techniques at our disposal:
- DNS prefetching
- Resource prefetching
- Prerendering
Now let’s dive into each of these separately.
DNS prefetching#section3
Whenever we know our users are likely to request a resource from a different domain than our site, we can use DNS prefetching to warm the machinery for opening the new URL. The browser can pre-resolve the DNS for the new domain ahead of time, saving several milliseconds when the user actually requests it. We are anticipating, and preparing for an action.
Modern browsers are very good at parsing our pages, looking ahead to pre-resolve all necessary domains ahead of time. Chrome goes as far as keeping an internal list with all related domains every time a user visits a site, pre-resolving them when the user returns (you can see this list by navigating to chrome://dns/ in your Chrome browser). However, sometimes access to new URLs may be hidden behind redirects or embedded in JavaScript, and that’s our opportunity to help the browser.
Let’s say we are downloading a set of resources from the domain cdn.example.com using a JavaScript call after a user clicks a button. Normally, the browser would have to resolve the DNS at the time of the click, but we can speed up the process by including a dns-prefetch directive in the head section of our page:
<link rel="dns-prefetch" href="http://cdn.example.com">
Doing this informs the browser of the existence of the new domain, and it will combine this hint with its own pre-resolution algorithm to start a DNS resolution as soon as possible. The entire process will be faster for the user, since we are shaving off the time for DNS resolution from the operation. (Note that browsers do not guarantee that DNS resolution will occur ahead of time; they simply use our hint as a signal for their own internal pre-resolution algorithm.)
But exactly how much faster will pre-resolving the DNS make things? In your Chrome browser, open chrome://histograms/DNS and search for DNS.PrefetchResolution. You’ll see a table like this:
This histogram shows my personal distribution of latencies for DNS prefetch requests. On my computer, for 335 samples, the average time is 88 milliseconds, with a median of approximately 60 milliseconds. Shaving 88 milliseconds off every request our website makes to an external domain? That’s something to celebrate.
But what happens if the user never clicks the button to access the cdn.example.com domain? Aren’t we pre-resolving a domain in vain? We are, but luckily for us, DNS prefetching is a very low-cost operation; the browser will need to send only a few hundred bytes over the network, so the risk incurred by a preemptive DNS lookup is very low. That being said, don’t go overboard when using this feature; prefetch only domains that you are confident the user will access, and let the browser handle the rest.
Look for situations that might be good candidates to introduce DNS prefetching on your site:
- Resources on different domains hidden behind 301 redirects
- Resources accessed from JavaScript code
- Resources for analytics and social sharing (which usually come from different domains)
DNS prefetching is currently supported on IE11, Chrome, Chrome Mobile, Safari, Firefox, and Firefox Mobile, which makes this feature widespread among current browsers. Browsers that don’t currently support DNS prefetching will simply ignore the hint, and DNS resolution will happen in a regular fashion.
Resource prefetching#section4
We can go a little bit further and predict that our users will open a specific page in our own site. If we know some of the critical resources used by this page, we can instruct the browser to prefetch them ahead of time:
<link rel="prefetch" href="http://cdn.example.com/library.js">
The browser will use this instruction to prefetch the indicated resources and store them on the local cache. This way, as soon as the resources are actually needed, the browser will have them ready to serve.
Unlike DNS prefetching, resource prefetching is a more expensive operation; be mindful of how and when to use it. Prefetching resources can speed up our websites in ways we would never get by merely prefetching new domains—but if we abuse it, our users will pay for the unused overhead.
Let’s take a look at the average response size of some of the most popular resources on a web page, courtesy of the HTTP Archive:
On average, prefetching a script file (like we are doing on the example above) will cause 16kB to be transmitted over the network (without including the size of the request itself). This means that we will save 16kB of downloading time from the process, plus server response time, which is amazing—provided it’s later accessed by the user. If the user never accesses the file, we actually made the entire workflow slower by introducing an unnecessary delay.
If you decide to use this technique, prefetch only the most important resources, and make sure they are cacheable by the browser. Images, CSS, JavaScript, and font files are usually good candidates for prefetching, but HTML responses are not since they aren’t cacheable.
Here are some situations where, due to the likelihood of the user visiting a specific page, you can prefetch resources ahead of time:
- On a login page, since users are usually redirected to a welcome or dashboard page after logging in
- On each page of a linear questionnaire or survey workflow, where users are visiting subsequent pages in a specific order
- On a multi-step animation, since you know ahead of time which images are needed on subsequent scenes
Resource prefetching is currently supported on IE11, Chrome, Chrome Mobile, Firefox, and Firefox Mobile. (To determine browser compatibility, you can run a quick browser test on prebrowsing.com.)
Prerendering#section5
What about going even further and asking for an entire page? Let’s say we are absolutely sure that our users are going to visit the about.html page in our site. We can give the browser a hint:
<link rel="prerender" href="http://example.com/about.html">
This time the browser will download and render the page in the background ahead of time, and have it ready for the user as soon as they ask for it. The transition from the current page to the prerendered one would be instantaneous.
Needless to say, prerendering is the most risky and costly of these three techniques. Misusing it can cause major bandwidth waste—especially harmful for users on mobile devices. To illustrate this, let’s take a look at this chart, also courtesy of the HTTP Archive:
In June of this year, the average number of requests to render a web page was 96, with a total size of 1,808kB. So if your user ends up accessing your prerendered page, then you’ve hit the jackpot: you’ll save the time of downloading almost 2,000kB, plus server response time. But if you’re wrong and your user never accesses the prerendered page, you’ll make them pay a very high cost.
When deciding whether to prerender entire pages ahead of time, consider that Google prerenders the top results on its search page, and Chrome prerenders pages based on the historical navigation patterns of users. Using the same principle, you can detect common usage patterns and prerender target pages accordingly. You can also use it, just like resource prefetching, on questionnaires or surveys where you know users will complete the workflow in a particular order.
At this time, prerendering is only supported on IE11, Chrome, and Chrome Mobile. Neither Firefox nor Safari have added support for this technique yet. (And as with resource prefetching, you can check prebrowsing.com to test whether this technique is supported in your browser.)
A final word#section6
Sites like Google and Bing are using these techniques extensively to make search instant for their users. Now it’s time for us to go back to our own sites and take another look. Can we make our experiences better and faster with prefetching and prerendering?
Browsers are already working behind the scenes, looking for patterns in our sites to make navigation as fast as possible. Prebrowsing builds on that: we can combine the insight we have on our own pages with further analysis of user patterns. By helping browsers do a better job, we speed up and improve the experience for our users.
Great article, with great technique and these things are really under discussed over the internet. Any 3-rd party JS powered libraries/solutions for that?
Hi Eimantas,
Thanks for your comments. I agree this is under discussed at this time. As far a JS libraries, you really don’t need one because enabling this is extremely simple based on meta tags.
Thank you @Santiago for taking the time to post the information which in invaluable. Posts like this make A List Apart one of my favorite go to place for getting updated.
Do we need how Google achieves the pretender for search results? Can’t seem to find their
Hi Henning,
I had the same question some time ago, and I didn’t find the tag either on their page. Unfortunately I can’t say how they are doing it, but it’s well documented that they do this for their search results, so for me is a little bit of a mystery.
I guess that’s one of the perks you get when your company is running the site as well as the browser 😉
Thanks for the great article! However – 😛 – I would like to play the devil’s advocate here. Where I live, out here in poor old rural Africa, data is hugely expensive. Hugely. I turned off Chrome’s “predict network actions to improve page load performance” to save costs. I often browse without pictures to save costs. Do you know how rubbish the net is without pictures? That’s how expensive data is…
Nobody knows what I’m going to do except me. My decisions cost money. I’d prefer it if someone else didn’t make costly decisions on my behalf. Fast page download is nice and all but watching the spinner is cheaper. I like the spinner. The spinner is a friend of mine.
Before going pedal to the metal on pre-rendering and such, wouldn’t it be better to ask the user if that is what they want? It’s pretty simple to adjust page code dynamically from there… Just saying… 😉
Thanks Santiago for that article. 🙂
It’s worth noting that none of these techniques is currently a standard. Ilya Grigorik is leading the standardization efforts at https://github.com/igrigorik/resource-hints which may result in syntax changes from what you show here, so it’s a good idea to keep an eye open on the emerging spec.
@Henning – I’m not sure if and how Google search preload/prerender search results, but Chrome doesn’t include any Google-only secret APIs.
@Gaétane – That’s a very good point and the reason these hints are now being specified as just that – hints. They can be overridden by the browser or by the users themselves, if the browser provides them with the tools to do that. Your data plan is yours. Preloading using a standard declarative mechanism rather than JS hacks will enable you to keep it that way.
@Gaétane You are right. Unfortunately is not only poor rural Africa, but most of the world is still suffering from more-than-slow Internet connections. These techniques put the entire focus on how to make the page load faster sometimes at the expense of data being fetched when’s not necessary.
As @Yoav mentioned, Google (specifically Ilya) is pioneering the specification for Resource Hints that should make this matter better. Resource Hints is not only for this type of stuff, but it defines a mechanism for the browser to “communicate” with the server and let it know what it needs. It’s a little more complicated than this, but I think that’s a fair simplification.
Hopefully two things will happen going forward: In one hand, Internet is going to get progressively better everywhere. On the other hand, standardization will allow users like you to have more control over the pages you visit.
Hmm… The first thing that popped into my mind was… that’s going to use my bandwidth for things I ‘may’ view. With each mb potentially costing me money on my cellular plan, I wouldn’t be too happy downloading stuff before I ask for it.
Just saying 🙂
Otherwise, a great concept.
@Jim, that’s correct. That’s why I tried to be very careful explaining the drawbacks of these methods if they are not used correctly.
Like in everything else, there’s no silver bullet to solve every single problem. Prebrowsing is another “tool” that web developers have to make their websites faster, but when used incorrectly it will be harmful.
Imagine a website that includes a 10MB image in it. As a user you have no control over that, and you’d be penalized if you open that page in your mobile phone. This is a developer’s decision (a bad one). Same thing happens with prebrowsing.
I would suggest there are some significant privacy issues with this proposal.
It opens up the possibility that users could be tracked through sites they don’t even visit, and would be a significant increase in the ability to monitor user behaviour.
Bear in mind that a large proportion of resources that slow down load times are advertising related.
As pointed out already, with more and more browsing on mobile devices with metered data – who wants to pay to download an advert, that can then track them, when they aren’t going to see it or visit the site that it is helping to pay for?
Thank you for the article! On the login page, if the welcome or dashboard page resources are prefetched or prerendered, how does that affect security especially if the user is not able to login?
Nice article! I have one concern though. How would these techniques affect the perceived bounce rate of sites? It is used to have an idea of how engaging a site is, if we use prebrowsing and not take this into consideration when analyzing the bounce rate we might end up with false data, since we might be counting requests that were never consciously made by the user. Anyhow, I’m sure techniques will emerge to make up for this.
Hey @Miriam, you made a good point! The good news is that trying to prerender a page creates a request from the browser to the server, so we are not bypassing any security in place. So if we are displaying a page that’s only visible to authenticated users, then prerendering will fail.
However, we might have a ‘shell’ page that we prerender and only the information specific to the user is loaded and secured. In this case, we can prerender the page (not the content displayed on it). It’s all about how we design our application.
@Manuel, Google Analytics already supports this, so they won’t count visits (or any other stat) coming from a prerendered page. However, this depends on each analytics provider. You can also instrument your site to act differently on prerendered pages. Here are more details on how Chrome handles all this: https://developers.google.com/chrome/whitepapers/prerender
Also, prerendering is still a work in progress and it hasn’t been standardized yet. Ilya Grigorik (from Google) is working on the standard (called Resource Hints). Go and check it out if you are curious.
Great article! I’ve used a JS plugin recently that prerenders pages on link hovers, which seems to work nicely. – http://css-tricks.com/add-page-transitions-css-smoothstate-js/
I was wondering does the prefetch happen at the same time as the initial page load delaying the time a user waits to see the page or does it occur when the initial page has finished loading?
Hi @Mika, the prefetch happens when the browser is idle. Prefetching or Prerendering doesn’t affect the current page load time.
Great article @Santiago, completely agree that we should look at rebuilding the approach to content/pages and I like the summary of browser options which are inbound.
We have recently launched our new site @tide_studio and have taken an (early stage) approach to this with our service section on the home page. The whole concept to our approach has not just been performance, but around the content ‘conversation’.
We have developed our approach with greater options, browser support, fully manageable via cms and the scope to iterate how/why it behaves for users.
We have just launched and this implementation is a baby step, but we will share in more detail.
Thanks for sharing @Kevin! I’m definitively interested to see how that turns out for you! Feel free to ping me in Twitter (@svpino) with updates.
Deal @Santiago, will do! (mitchinator / tide_studio)
Great article Santiago Valdarrama… its save lots of waiting time of users whenever they click on button to fetch some internal or external URL or data.
Thanks…
@Santiago,
From my understanding DNS prefetching occurs right away, while the other two do occur during idle time.
Very interesting technique, thanks for sharing. I have a question about it though: what would happen if one would dynamically (via javascript) include a prefetch link into the dom? Would that have any effect?
Great article Santiago ! I learned a lot while reading it. However I was wondering, for a resource prefetching if this would work ?
<link rel=”prefetch” src=”src/js/example.js”>
Hi @powrsurg, you are right, but it really depends on the browser’s implementation. Remember all these techniques are only hints for the browser. That doesn’t mean that browsers are going to do anything on the first place. However, currently Chrome does execute the DNS prefetch right away.
@fchristant, Interesting question. I’m not sure, but I guess I’ll try to see if that works.
@ChucKN0risK, that should work.
Just thinking, isn’t dns-prefetch a misnomer since it requires a protocol? It’d only really be a dns-prefetch if it was just resolving the domain into an IP …
Or maybe I have not had enough coffee yet this morning given a very long night ….
@Santiago: great writeup! One nitpick: prefetch, as currently implemented, is meant to be used for resources required by the “next navigation”, which means that they will be downloaded with low priority (or not at all), after all other resources have finished loading. As a result, prefetch should not be used for hinting critical resources on the *current page*. As it happens, this is also the reason why Chrome introduced rel=subresource, but subresource has its own set of problems… so I wouldn’t recommend that either.
TL;DR: Today, use dns-prefetch for current and next pages; use prefetch and prerender to fetch resources for next navigation.
Thanks @Ilya! Great having your 2 cents here!
Great article and I definitely feel it’s a topic that is seldom discussed. I can see this technique becoming more prominent as bandwidth costs begin to fall around the world.
Thank you Santiago for the great and inspiring article! I am not a tech guys, but it is really interesting that I am enjoying the benefits of these techniques without knowing them until I read your article. I think the purpose of prebrowsing is to increase user experiences. However, as Grange mentioned on his post that data is hugely expensive in old poor rural Africa (probably not only Africa but also in many other countries), the priority concern for them will be the costs and then the user experience. It may be a good idea to give users some control or options over the pages/websites they visit.
As a marketing major student, this article provides me some good ideas on how to predict consumer behavior. Just like prebrowsing, gathering data is the first and probably the most important thing to do. The data we need for true customer insight is likely from multiple data sources. Next, careful customer analytics will allow companies to predict customer behavior and use the insight to improve their business including efficiency, retention rate, satisfaction, loyalty etc.
The problem is that everything has two sides. For instance, prerendering is the fastest among the three techniques, but it is also the most risky and costly one. Similarly, companies can gather a lot of customer data, but the data is not used as effectively as they should be due to the day-to-day challenge. Anyway, I strongly believe that these problems will be solved in the near future as our technologies become more mature.
I just found this via the “summer reading” article. Great read :). I actually did my own writeup on prebrowsing/preloadable content and specifically using a Rails gem to do it programmatically. Let me know what you think :).