The Perfect 404

Oops. Something went wrong. You’re not sure what — was it you? Was it the website? What do you do now?

Welcome to the world of the Error 404 page. You’ve requested a page — either by typing a URL directly into the address bar or clicking on an out-of-date link and you’ve found yourself in the middle of cyberspace nowhere. A user-friendly website will give you a helping hand while many others will simply do nothing, relying on the browser’s built-in ability to explain what the problem is. We can do better than that, can’t we?

I won’t go into details about how you set your server up to deliver a custom 404 page — instead, I’ll refer you to these articles:

• Creating a Custom 404 Error Page (4 Guys From Rolla)
• Creating Custom Error Messages in Apache (4WebHelp)

I will, however, suggest strategies for building a custom 404 page that makes the most of an otherwise lost cause.

To get started, we need to examine the most common reasons that people find themselves on a 404 page:

• a mis-typed URL (or an out-of-date bookmark/favourite)
• a search-engine link that is out-of-date
• an internal broken link that the webmaster hasn’t noticed

They all amount to the same thing, but they need to be handled slightly differently — the 404 message needs to be customized for each eventuality. There are some tricks that you can employ that apply to all scenarios, but my first suggestion is simple…

Don’t point the finger#section2

Tell them what went wrong, by all means, but don’t admonish the user, even if you know it’s their fault! The phrases, “might have” and “possibly” are good ones to use here. Don’t get off on the wrong foot with this visitor to your site — you might yet turn this problem around.

404 Must-haves#section3

As well as the “something went wrong” text, you should ensure that your error page has the following:

• A link to the site map (if you have one) and the home page. This is the easiest way for users to bail out. This no-brainer requires no clever scripting.
• A search box. If you have a site search, add it to your 404 page. If you don’t have a site search and are in the habit of generating 404 errors, perhaps you should get one.
• A distinctly minimalist look. Avoid putting all your standard site navigation on this page. You should aim to remove distractions. Besides, insisting on including a complete site navigation strip may present a maintenance overhead (your 404 page can easily lag behind the rest of the site if it is not dynamically updated with the rest of your site and the last thing you want is to have navigation on the 404 that is no longer relevant/working. Oh the irony!)

Also be sure to cut the jargon. I mean, we can talk about 404s here, right? We’re in good company. But 60-year-old Doris who got a broken link while browsing a knitting site won’t have a clue what a 404 error is. If you want to use the phrase “Error 404,” do it subtly — do it as a footnote, a nod to those who understand your freaky-deaky web speak.

Now, let’s look at how you can make the 404 work for you, instead of against you.

Let’s be intelligent about this#section4

At this point, I should point out that this may require a certain amount of JavaScript (you may not be able to use server-side scripting to do this work, depending on your server set-up). So be sure to use the <noscript> to serve up a suitable message for people who have disabled scripting. If you can use server-side scripting, that would be preferable — no accessibility or browser support issues — so adapt the suggested code as appropriate.

First, you’ll need to set up a few variables:

var strReferrer=document.referrer.toLowerCase();
var blnSearchReferral = false;
var blnInsiteReferral = false;
var str="";
var strSite = "";

Now, how are we going to use these?

The mis-typed URL#section5

A mis-typed URL (or out-of-date bookmark) will have no referrer, so your code to identify this scenario should look something this:

if (strReferrer.length==0)
  {
  str+='We think you will find one of the following »
    links useful:<\/p>';
  str+='<a href="\/home.php"><img src="/images/ »
    home.gif" alt="Home Page" width="100" height="30" »
    \/> <\/a>';
  str+='<a href="\/site-map.php"><img src="/images/ »
    site-map.gif" alt="Site Map" width="100" height= »
    "30" \/><\/a>';
  str+='<hr \/>';
  str+='<p><strong>You may not be able to find the »
    page you were after because of:<\/strong><\/p>';
  str+='<ol type="a">';
  str+=' <li>An <strong>out-of-date bookmark\/favorite »
    <\/strong><\/li>';
  str+=' <li>A search engine that has an <strong>out- »
    of-date listing for us</strong><\/li>';
  str+=' <li>A <strong>mis-typed address</strong><\/li>';
  str+='<\/ol>';
  document.write(str);
  }

The out-of-date search engine referral#section6

If there is a referrer value, we can look for an instance of specific search engines (and you can obviously tweak this to your own taste). Then we can split the search parameters up, look for significant matches to a list of terms you want to intercept and suggest a page that is probably appropriate to that search:

if (strReferrer.length!=0) {
  if ((strReferrer.indexOf(".looksmart.co")>0)||
  (strReferrer.indexOf(".ifind.freeserve")>0)||
  (strReferrer.indexOf(".ask.co")>0)||
  (strReferrer.indexOf("google.co")>0)||
  (strReferrer.indexOf("altavista.co")>0)||
  (strReferrer.indexOf("msn.co")>0)||
  (strReferrer.indexOf("yahoo.co")>0))
  {
    blnSearchReferral=true;
    //get site domain — split at the first forward-slash
    var arrSite=strReferrer.split("/");
    // now find search parameters
    var arrParams=strReferrer.split("?"); 
    var strSearchTerms = arrParams[1];
    arrParams=strSearchTerms.split("&");
    
    strSite=arrSite[2];
    var sQryStr="";
    
    //define what search terms are in use by the different engines
    var arrQueryStrings = new Array();
    arrQueryStrings[0]="q=";  //google, altavista, msn
    arrQueryStrings[1]="p=";  //yahoo
    arrQueryStrings[2]="ask=";  //ask jeeves
    arrQueryStrings[3]="key=";  //looksmart
    
    for (i=0;i<arrParams.length;i++)
    //loop through all the parameters in the referring page’s URL
    {
      for (q=0;q.indexOf(sQryStr)==0)
      {//we’ve found a search term!
      strSearchTerms = arrParams<i>;
      strSearchTerms = strSearchTerms.split(sQryStr);
      strSearchTerms = strSearchTerms[1];
      strSearchTerms = strSearchTerms.replace("+", " ");
      }
    }
  }
  //Tell the visitor what site is at fault, what the 
  //search terms were
  document.write ("<p>You did a search on <strong> <a href='" + strReferrer + "' target='_blank'>" + strSite + "</a> </strong> for "<strong>"+ strSearchTerms + "</strong>". However, their index appears to be out of date.</p> 
  <h2>All is not lost!</h2>
  
  <p>We think that the following page(s)on our site will be able to help you:</p>");

You can now add in some lines to identify key search phrases that you don’t want to be lost. For example, say you’re getting good search results for the words “electronics” and “widgets” but you’ve just moved pages around in your site — no good losing all those existing Gooogle referrals, is it?

if (
  (strSearchTerms.indexOf("widgets")>=0)||
  (strSearchTerms.indexOf("electronics")>=0)
  )
  {
    document.write("<a href='/cool-widgets.htm'>Our excellent widgets page</a><br />");
    }
  }
}

Of course, if you have an internal site search, you could use the search parameters found in the URL to generate a search automatically, rather than this very manual process. However, the manual approach may be preferred (otherwise you may only be adding another hit-and-miss search process to one that already failed).

The broken (in-site) link#section7

With the errant search engine referrals taken care of, we now need to look at referrals that are not from a search engine (or at least not a search engine you’ve chosen to look for). We’ll need to add some more conditions:

if (!blnSearchReferral) {
  strSite = strReferrer;
  strSite = strSite.split("/");
  strSite = strSite[2];
  document.write("<p>You were incorrectly referred to this page by: <strong><a href='" + strReferrer + " 'target='_blank'>" + strSite + "</a></strong> <br />We suggest you try one of the links below: </p>");
  }

… those links being the site map and home page mentioned right up front.

What if your own site is the problem?#section8

You cannot say in your 404 page “This site had an incorrect link” when the errant referrer is actually one of your own. In this instance, you might need to change the language to admit some guilt!

blnInsiteReferral =((strReferrer.indexOf("http://www.mysite.co.uk")>=0)||
(strReferrer.indexOf("http://www.myothersite.com") >=0))
if (blnInsiteReferral) {
  document.write("<p>This one’s down to us! Please accept our apologies for this — we’ll see to it that the developer responsible for this broken link is given 20 lashes (but only after he or she has fixed this problem).</p>");
}

Fixing the problem#section9

>So, we’ve provided routes out of the 404 black hole. But have we actually fixed anything? No. We do have some options though, as we know the page requested and the referring page (if indeed there was a referrer). We could capture this information in a database, either automatically or via a manual “report this broken link” button offered to the user when they hit the 404 page. The latter should cut down on noise, and the hope is that you’ll only get to hear about the most important broken links. How you go about fixing them from here on in is up to you.

Related links#section10

To see this advice in action, try the following examples hosted at Accessify and A List Apart:

[Note: the accessify.com pages seem to have been lost to time. —Ed.]

• paste this in to your address bar to show the behavior for an out-of-date bookmark: http://www.accessify.com/err.asp
• a broken link
• mimicking a Google search [intermediate page required to trigger]

Download the example error 404 page described above (and amend to suit your own needs).