“In a business environment where marketers demand an accountable performance from every web page, it’s time to put aside the assumed expertise of design and copy “gurus.”?”
Doesn’t that cover pretty much the entire ALA readership?
“[…] let our readers show us which designs work best, and which copy works best.”
Anyone remember the car that Homer Simpson designed?
“[…] the end result is that we will become much better web designers and writers.”
“Better” is a very subjective term, but I’d like someone to show me how the triumph of design B could possibly make us better or worse designers and writers.
Copy & paste the code below to embed this comment.
Dana Kowalski
as said above if this is a layout test, then copy needs to be the same. If this is a copy test then layout needs to be the same for adequate controls in testing. This isn’t really a decision made about ‘designs’ here because the call to action and the copy is different on each item.
I would also like to see the test with all three including the click now in the same positioning, and a test with none of them including the click now. I think that would make a more compelling article than this (nothing personal to our author who put in his time and effort).
btw I thought C was the most visually appealing while still providing the baseline information for the site within a decent amount of breathing room.
Copy & paste the code below to embed this comment.
Dana Kowalski
as a followup to my previous post above here is my thoughts on all the designs:
design A: I think this one is functional and provides a good choice. Personally the call to action not being immediately visible, and action required at the bottom of the page you could argue the negatives of.
design B: I actually thought this was the worst of the 3. there is no white space, it comes off as too cramped and riding up on me. It needs some room to breath in my opinion and the right columned text size, while I like it, would be the first thing I would receive as a complaint [:(].
design C: I thought this was the most visually compelling, to me, of the three anyhow. It traded off and compromised well with visual appeal, breathing room, and still providing instant access to the neccessary information. Again the call to action could be questioned. I believe this choice would always be a designers choice, whether a functional end user choice is another discussion.
Copy & paste the code below to embed this comment.
Rick Jensen
While the use of white-space, font color, and layout contribute greatly in any project, could the answer to the swing in sales be security?
I stay as far away from on-line store fronts in the development world, but I do look for the visual cues when I am a consumer.
A :
-No lock on credit card field
-No Verisign logo
-No security policy info
B :
-Lock on credit card field
-Verisign logo
-Authenticated Verisign symbol
-No security policy info
C :
-Lock on credit card field
-No Verisign logo
-Authenticated Verisign symbol
-Security policy info
Is that the full answer?
No.
The layout of C may have played a role in the flux of sales, but we may see the comlicated formula of online sales and marketing take a turn when the “sense of security” is included in the equation
i am sure version c could be improved significantly. i wonder why the form has been put into the hot spot and not the example. would be interesting to see how conversion changes when the columns were switched. and it would be fair to give version c the same header and introduction as version b without mentioning the price etc.
Copy & paste the code below to embed this comment.
lydia dove
I am working on a Technical Degree in Web Design and I eat this kind of information daily trying to digest it out as a style statment. I saw no difference in overall design in B or C except the column change. What am I missing? My goal is to be a w3 compliant designer and use CSS for everything but I keep getting hung up with wanting to put design elements in where they don’t belong. Any advise? From “Stranded in Wyoming”
Copy & paste the code below to embed this comment.
Marcus F
This is digressing a bit from the article, but I must say that the idea for the site is probably by far the most disgusting thing I’ve seen in my entire life. It pretty much made me sick. Hanging out people like that…
Now I see why this might be a great buisness-model, with the parents and all, but holy crap. It really hurts to see an actual site like this.
I agree that the point that design choices matter is stating the obvious. However I personally took the articles’ overal point to be that a designer needs to test there layouts thoroughly in a practical, analytical and possibly statistical manner. I think this advice is apt as myself and most designers I know tend to test there layouts by showing to a few people and discussing their merits rather than actually running tests on them.
That has got me thinking about how I shall test layouts in future.
I do agree that the specific analysis would need to be more in depth than the article presents to be truly useful. You don’t want to just know that B got a bit more sales than A and many more than C, but also WHY that is the case.
Copy & paste the code below to embed this comment.
Peter Wardley-Repen
Never mind the design choices, what about the ethical choices? I find it disturbing that you would design a website for, effectively, vigilantes.
I’m no more in favour of the people your example site names than the next guy (sorry about the wording, I’m allowing for filters), but there have been enough cases of mistaken identity leading to innocent people getting hurt that sites like this are a really bad idea. There are established legal channels for dealing with this sort of thing, and making sites like this is just pandering to vigilantism – and worse, profiting from it.
However valid the design issues, a bad choice of example.
but the Sample Report should have been on the left and the form on the right. Bah! scrap that idea there is too much on screen! Try putting the form on another page?
Copy & paste the code below to embed this comment.
William Bentley 2
I liked the article, but it was not scientific. No real control, plus the data sample was too small. I think the best layout would have been a combination of B and C, with the prominent sales “copy” (Click Here) prominently displayed. In all three cases the yellow button at the bottom of the screen did little to impact first impressions. The call-to-action is the key element in Item B that likely did the work.
Copy & paste the code below to embed this comment.
Chris Eilers
Leaving aside the question of the tastefulness or otherwise of the web page in question, is this article really on the level?
Out of curiosity I googled my way to nationalalertregistry.com, entered the zip code of 32225 and got….Version A. And just in case nationalalertregistry.com were still doing user testing a la comment #37, I tried accessing the same nationalalertregistry.com page via two other browsers. Same result.
Copy & paste the code below to embed this comment.
roland benedetti
I still don’t understand why you don’t write this kind of article using figures from an hotel booking web site or anything more usual and less … subject to questions …
anyway :
B/ Is ugly. No space. Not readable. May be there is good idea in it, but the design itself is a mess i thing. Of course it improves conversion because of the “Click here” button. You could have the same result keeping the A design and simply adding this link at the top …
C/ Is not very good from a design point of view but the idea is ok. It would ask for a redeisgn. You can’t really have any conclusion on the figures you give if you don’t know more (duration of the test, target list, new users / recuring users …)
My conclusion reading this article, as many other comments, is that all is in the title and not more ;-( but it can be interesting for people who are not aware of that (i thing ALA readers are …)
Subject matter aside, am I the only one who thinks that all 3 of these designs are amateur and hideous?
Does fact that one terrible design tested better than two other terrible designs make this a valuable study of web design (especially when each version used completely different verbiage) ?
I could go into my garage and bottle 3 different homemade wines. I could then go down to the vacant lot beside the local liquor store and conduct a taste test with some ‘users.’ One of my wines would score higher than the other two. What would this prove? Obviously it proves that the wrong wine label design can cripple a wine. To be perfectly analogous with the subject matter I should probably note that these particular wines are being used prepping the user for a ‘Bum Fight’ which we are taping at the same location.
Naturally these would be some valuable findings, and I would submit them to Wine Spectator magazine, who, being well regarded by vineyards and spectators alike would surely put the story on the cover with a provocative headline.
Despite what percentages say, I do really prefer PageC. PageA is also better than B, since B does really make me sick when I try to read it: line-height and white space in general are definitely too poor. This is the most important thing to be noticed in my opinion, more than the 2 or 1 coloumn layout. In fact I think that the best design might be a re-designed C version. Two changes might be:
1) Please take away that damn FREE word, i’m sure people associate it with SPAM. I do. You do. We do. They do. So it’s a stupid bug that corrupts the design of the page.
2) The 2 coloumns layout is ok, but why should they be put this way? As the natural flow of the reader tries to find the sample first, just put it on the left. I think it’s quite obvious, isnt it?
Of course many other changes are to be made, but these are really important im my opinion.
PS: When I see a “click here” link the very first thing I do is laughing. Guess why.
Do you think the fact pageC mentions the cost straight away, whereas the other two don’t mention cost until you have been sold what you get in the report, has an effect?
I think the space and look on pageC is easier to read than the other two, but I would stick to selling the report first and having the form underneath. I also think the headline in pageC makes the purpose of the page clearer. What about keeping the same text as B but laying it out with the spacing from C.
I think this is interesting but, I’m a bit put off by your scare mongering to make money. I just hope you don’t make mistakes and send people to the wrong addresses like has happened in this country(UK).
Does fact that one terrible design tested better than two other terrible designs make this a valuable study of web design (especially when each version used completely different verbiage)?
This is not about what factors make a good design. It is not about whether we, as authors, prefer one design to another.
It is about how we find which designs work best for any given site (remembering that different sites have different demographics, so require different approaches).
The aim behind this article (as far as I can tell) was not to say “B is a wonderful design”, nor was it to promote vigilante-ism or to disgust readers by the choice of material. It was to show a real example of A/B testing and how the results might differ significantly from what we expect.
We could now discard the A and C designs, and refine B into B1 and B2, and see which of those performs best, and use that as the basis to improve the design further. Don’t think that the game is over just because we have completed Level 1.
To be perfectly analogous with the subject matter I should probably note that these particular wines are being used prepping the user for a “Bum Fight” which we are taping at the same location.
I can’t see having 3 different page designs, active at different times of the year (thus the market can shift significantly during those times) as yielding results worthy of legitimate note.
We all want to analyze how visitors navigate our website, and there are (somewhat limited) tools available to do so to an extent, but I feel that any site which doesn’t necessarily bombard the user with data, and instead takes an approach in which navigation and information are minimalized, will have visitors who might not make a purchase, but when they do, may both do so quicker, and return more often; due to an ease of use factor.
The article makes a good general point that design matters in drawing a user in/pushing them away, but comparing the three designs, as mentioned before, isn’t really fair.
The other question I’d ask is whether it’s actually fair to compare the performance of the three when we don’t know what controls were put on the tests. The three designs presumably weren’t all live at once, so who’s to say one design wasn’t just subject to a slow day/week of sales?
What other marketing was going on while each design was up on the site that may have affected results?
The designs change too much and (barring more info from the author) seem to have too few checks over the environment in which they were competing to make any real conclusions on the merits of each.
The other question I’d ask is whether it’s actually fair to compare the performance of the three when we don’t know what controls were put on the tests. The three designs presumably weren’t all live at once, so who’s to say one design wasn’t just subject to a slow day/week of sales? What other marketing was going on while each design was up on the site that may have affected results?
The usual way to carry out A/B testing is to have both versions live at the same time. Roughly half the users will be served one version of the page, and half the other. That should mitigate against the sort of other factors you mention.
I think B would be even better if the layout were more like A. Needs more whitespace. Layout C is just unnecessarily complex. The process of deciding on a small purchase is sequential: Read what it is, find out how much, purchase. But two columns forces the user to juggle all three things as soon as the page loads. You’re asking to fill out a form to buy something before it’s even clear what you’re selling.
However, given some of the (apparently) misinformed comments I think all three designs have a more serious flaw: It’s not clear what the product is. First, let’s make it clear that I know nothing about this company except what’s in the screenshots, and I don’t care if they exist or not. But if I understand the copy right (esp the “Background” section seen on the first design) this “report” is a compilation of publically available information that law enforcement agencies must provide to anyone who ask, in accordance with child protection laws in the US. Some states do a decent job presenting the information online by themselves. Others do a bad job, thus creating a market for third-party sites like this.
Those may or may not be good or effective laws (take it up with your member of congress) but the result is certainly not some random “creepy guy” blacklist compiled by fly-by-night spammers as some have implied. Calling the author/designer unethical seems both uninformed and unfair.
This article really missed the mark by not trying to really analyze why B was slightly better. As others mentioned here it probably had something to do with the testimonials being more avaiable and the huge “Click Here” steering the user past the jumble of content. Though personally I think the click here is just a cop out for a bad design with no clear path for the eye to follow. Sure C is confusing but so are the others—they just are better because at least there is a simple down direction to follow.
I agree with the article that real testing is great but really… how often can you get a client to pay for that? This article could have served better if it not only tested but analyzed the whys. For at least then we could walk away with a bit more understanding for use the next time a client won’t pay for design user testing.
This is sound advice for ANY business in ANY industry. Live testing is the only way to know what really works, and everyone should be open to the idea that release #1 may not be (and probably isn’t) the most effective version.
For the record, I used live testing in a phone card distribution business, and was able to come up with a protocol that allowed sales to increase linearly (to keep up with cash flow) in an almost unbounded fashion. The results after 18 months of this gameplan? 130% increase in sales, 400% jump in profits, 81% jump in per-unit profit.
When I read something like this, it just serves to drive home a point that I was lucky enough to have firsthand experience with. I hope everyone who reads this sees the value here.
I understand the point the author is trying to make but the methodology is appalling and acts to counter, the useful point that I think is lurking in the article.
To make a test meaningful you can only alter one variable. How can you tell what caused the difference in sales if both copy and layout was changed. As an earlier commenter suggested, perhaps it was the copy that had the real effect, not the layout as the author implies.
Copy & paste the code below to embed this comment.
Jon Smith
The major difference to the results was copy changes, not design. The authow even stated as much in the opening: “Version B follows the same basic layout, but we made some minor copy changes.”
To my eyes all 3 options looked confusing….didnt know where to look or what different area’s did.
Hire a professional web designer to actually design the site rather than tinkering with slight variations…. THEN see the results.
When I clicked the Marketing Experiments’ Journal link in our author’s profile it took me to alovely white background page with one sentence on it… No website is configured for this URL.
The author’s condescending remark, “I know they won’t win any design awards. But they are functional and familiar” betrays the offensive implication that designers simply decorate web sites with pretty graphics, with no understanding of what they are doing.
The article is not ‘provocative’ because it is suggesting something radical or novel, it is provactive because of it’s crass arrogance and ignorance.
Surely in 2006 we’re not still debating the value of design! The purpose of design (whether in print or web) is to aid the communication of ideas and information, and to create desire.
I would be amazed if an IA, information designer or graphic designer had been anywhere near the example pages that were shown. All three were terrible – it’s a wonder that site made any sales at all!
I am all for live testing (under rigorous conditions),,, but does the author of this article (which I have to say, has no place on this illustrious site) seriously believe that as designers we are just poking around in the dark hoping for the best?!
Copy & paste the code below to embed this comment.
P R
First: Back to the basic purpose of the article: improving website design. Still pertinent and still needing to be examined…even in this lofty year of 2006! In fact, what seems to be missing in relating to the comparision test comments—is the realization of what is the difference in the effects between A, B, C.
Too many of the comments are concerned with details. Without a doubt, the best version was site B. Why? Structurally it was ergonimically appreciated visually (meaning eyeballroll) and did not ‘yank’ the head down as was version A’s affect…which to a tired neck, is a complaint. Further, a minor frustration that the core of the info which immediately ‘weights’ seems to be sinking into the bottom of the computer screen.
Test it yourself. Where are your eyes immediately drawn to before you reavert to search the site? Looking right down at the bottom of the screen. Now you understand the effect of the neck being ‘yanked’.
In version B, the bulk that the eye first seeks is better positioned to the upper left, and is more effortless to view overall.
Copy & paste the code below to embed this comment.
clay davis
forgive me if this was mentioned already but I think b was most succesful simply because of the “want names pictures and address” link near the top. who wouldn’t avoid mucking through such a bad layout if they could click on a link and get what they really wanted? the rest of the design choices in my opinion are irrelevent.
74 Reader Comments
Back to the ArticlePhil Stewart-Jones
“In a business environment where marketers demand an accountable performance from every web page, it’s time to put aside the assumed expertise of design and copy “gurus.”?”
Doesn’t that cover pretty much the entire ALA readership?
“[…] let our readers show us which designs work best, and which copy works best.”
Anyone remember the car that Homer Simpson designed?
“[…] the end result is that we will become much better web designers and writers.”
“Better” is a very subjective term, but I’d like someone to show me how the triumph of design B could possibly make us better or worse designers and writers.
Dana Kowalski
as said above if this is a layout test, then copy needs to be the same. If this is a copy test then layout needs to be the same for adequate controls in testing. This isn’t really a decision made about ‘designs’ here because the call to action and the copy is different on each item.
I would also like to see the test with all three including the click now in the same positioning, and a test with none of them including the click now. I think that would make a more compelling article than this (nothing personal to our author who put in his time and effort).
btw I thought C was the most visually appealing while still providing the baseline information for the site within a decent amount of breathing room.
Dana Kowalski
as a followup to my previous post above here is my thoughts on all the designs:
design A: I think this one is functional and provides a good choice. Personally the call to action not being immediately visible, and action required at the bottom of the page you could argue the negatives of.
design B: I actually thought this was the worst of the 3. there is no white space, it comes off as too cramped and riding up on me. It needs some room to breath in my opinion and the right columned text size, while I like it, would be the first thing I would receive as a complaint [:(].
design C: I thought this was the most visually compelling, to me, of the three anyhow. It traded off and compromised well with visual appeal, breathing room, and still providing instant access to the neccessary information. Again the call to action could be questioned. I believe this choice would always be a designers choice, whether a functional end user choice is another discussion.
Rick Jensen
While the use of white-space, font color, and layout contribute greatly in any project, could the answer to the swing in sales be security?
I stay as far away from on-line store fronts in the development world, but I do look for the visual cues when I am a consumer.
A :
-No lock on credit card field
-No Verisign logo
-No security policy info
B :
-Lock on credit card field
-Verisign logo
-Authenticated Verisign symbol
-No security policy info
C :
-Lock on credit card field
-No Verisign logo
-Authenticated Verisign symbol
-Security policy info
Is that the full answer?
No.
The layout of C may have played a role in the flux of sales, but we may see the comlicated formula of online sales and marketing take a turn when the “sense of security” is included in the equation
raphaele giordan
i can’t even understand that the designer accepted this contract, so how could i understand that he disuss about the commercial impact of his design !
Gerald Steffens
i am sure version c could be improved significantly. i wonder why the form has been put into the hot spot and not the example. would be interesting to see how conversion changes when the columns were switched. and it would be fair to give version c the same header and introduction as version b without mentioning the price etc.
lydia dove
I am working on a Technical Degree in Web Design and I eat this kind of information daily trying to digest it out as a style statment. I saw no difference in overall design in B or C except the column change. What am I missing? My goal is to be a w3 compliant designer and use CSS for everything but I keep getting hung up with wanting to put design elements in where they don’t belong. Any advise? From “Stranded in Wyoming”
Marcus F
This is digressing a bit from the article, but I must say that the idea for the site is probably by far the most disgusting thing I’ve seen in my entire life. It pretty much made me sick. Hanging out people like that…
Now I see why this might be a great buisness-model, with the parents and all, but holy crap. It really hurts to see an actual site like this.
Leo Pitt
I agree that the point that design choices matter is stating the obvious. However I personally took the articles’ overal point to be that a designer needs to test there layouts thoroughly in a practical, analytical and possibly statistical manner. I think this advice is apt as myself and most designers I know tend to test there layouts by showing to a few people and discussing their merits rather than actually running tests on them.
That has got me thinking about how I shall test layouts in future.
I do agree that the specific analysis would need to be more in depth than the article presents to be truly useful. You don’t want to just know that B got a bit more sales than A and many more than C, but also WHY that is the case.
Peter Wardley-Repen
Never mind the design choices, what about the ethical choices? I find it disturbing that you would design a website for, effectively, vigilantes.
I’m no more in favour of the people your example site names than the next guy (sorry about the wording, I’m allowing for filters), but there have been enough cases of mistaken identity leading to innocent people getting hurt that sites like this are a really bad idea. There are established legal channels for dealing with this sort of thing, and making sites like this is just pandering to vigilantism – and worse, profiting from it.
However valid the design issues, a bad choice of example.
Johan De Silva
but the Sample Report should have been on the left and the form on the right. Bah! scrap that idea there is too much on screen! Try putting the form on another page?
William Bentley 2
I liked the article, but it was not scientific. No real control, plus the data sample was too small. I think the best layout would have been a combination of B and C, with the prominent sales “copy” (Click Here) prominently displayed. In all three cases the yellow button at the bottom of the screen did little to impact first impressions. The call-to-action is the key element in Item B that likely did the work.
Chris Eilers
Leaving aside the question of the tastefulness or otherwise of the web page in question, is this article really on the level?
Out of curiosity I googled my way to nationalalertregistry.com, entered the zip code of 32225 and got….Version A. And just in case nationalalertregistry.com were still doing user testing a la comment #37, I tried accessing the same nationalalertregistry.com page via two other browsers. Same result.
So what’s going on here?
roland benedetti
I still don’t understand why you don’t write this kind of article using figures from an hotel booking web site or anything more usual and less … subject to questions …
anyway :
B/ Is ugly. No space. Not readable. May be there is good idea in it, but the design itself is a mess i thing. Of course it improves conversion because of the “Click here” button. You could have the same result keeping the A design and simply adding this link at the top …
C/ Is not very good from a design point of view but the idea is ok. It would ask for a redeisgn. You can’t really have any conclusion on the figures you give if you don’t know more (duration of the test, target list, new users / recuring users …)
My conclusion reading this article, as many other comments, is that all is in the title and not more ;-( but it can be interesting for people who are not aware of that (i thing ALA readers are …)
Zach Harkey
Subject matter aside, am I the only one who thinks that all 3 of these designs are amateur and hideous?
Does fact that one terrible design tested better than two other terrible designs make this a valuable study of web design (especially when each version used completely different verbiage) ?
I could go into my garage and bottle 3 different homemade wines. I could then go down to the vacant lot beside the local liquor store and conduct a taste test with some ‘users.’ One of my wines would score higher than the other two. What would this prove? Obviously it proves that the wrong wine label design can cripple a wine.
To be perfectly analogous with the subject matter I should probably note that these particular wines are being used prepping the user for a ‘Bum Fight’ which we are taping at the same location.
Naturally these would be some valuable findings, and I would submit them to Wine Spectator magazine, who, being well regarded by vineyards and spectators alike would surely put the story on the cover with a provocative headline.
Come on, ALA, show some editorial control.
luca rossi
Despite what percentages say, I do really prefer PageC. PageA is also better than B, since B does really make me sick when I try to read it: line-height and white space in general are definitely too poor. This is the most important thing to be noticed in my opinion, more than the 2 or 1 coloumn layout. In fact I think that the best design might be a re-designed C version. Two changes might be:
1) Please take away that damn FREE word, i’m sure people associate it with SPAM. I do. You do. We do. They do. So it’s a stupid bug that corrupts the design of the page.
2) The 2 coloumns layout is ok, but why should they be put this way? As the natural flow of the reader tries to find the sample first, just put it on the left. I think it’s quite obvious, isnt it?
Of course many other changes are to be made, but these are really important im my opinion.
PS: When I see a “click here” link the very first thing I do is laughing. Guess why.
barry ramsay
Do you think the fact pageC mentions the cost straight away, whereas the other two don’t mention cost until you have been sold what you get in the report, has an effect?
I think the space and look on pageC is easier to read than the other two, but I would stick to selling the report first and having the form underneath. I also think the headline in pageC makes the purpose of the page clearer. What about keeping the same text as B but laying it out with the spacing from C.
I think this is interesting but, I’m a bit put off by your scare mongering to make money. I just hope you don’t make mistakes and send people to the wrong addresses like has happened in this country(UK).
Stephen Down
This is not about what factors make a good design. It is not about whether we, as authors, prefer one design to another.
It is about how we find which designs work best for any given site (remembering that different sites have different demographics, so require different approaches).
The aim behind this article (as far as I can tell) was not to say “B is a wonderful design”, nor was it to promote vigilante-ism or to disgust readers by the choice of material. It was to show a real example of A/B testing and how the results might differ significantly from what we expect.
We could now discard the A and C designs, and refine B into B1 and B2, and see which of those performs best, and use that as the basis to improve the design further. Don’t think that the game is over just because we have completed Level 1.
Jeffrey Zeldman
Zach Harkey “said”:http://www.alistapart.com/comments/designcancripple?page=6#55 …
Heh. You said “Bum Fight.”
Aaron Martone
I can’t see having 3 different page designs, active at different times of the year (thus the market can shift significantly during those times) as yielding results worthy of legitimate note.
We all want to analyze how visitors navigate our website, and there are (somewhat limited) tools available to do so to an extent, but I feel that any site which doesn’t necessarily bombard the user with data, and instead takes an approach in which navigation and information are minimalized, will have visitors who might not make a purchase, but when they do, may both do so quicker, and return more often; due to an ease of use factor.
Adam Perfect
The article makes a good general point that design matters in drawing a user in/pushing them away, but comparing the three designs, as mentioned before, isn’t really fair.
The other question I’d ask is whether it’s actually fair to compare the performance of the three when we don’t know what controls were put on the tests. The three designs presumably weren’t all live at once, so who’s to say one design wasn’t just subject to a slow day/week of sales?
What other marketing was going on while each design was up on the site that may have affected results?
The designs change too much and (barring more info from the author) seem to have too few checks over the environment in which they were competing to make any real conclusions on the merits of each.
Stephen Down
The usual way to carry out A/B testing is to have both versions live at the same time. Roughly half the users will be served one version of the page, and half the other. That should mitigate against the sort of other factors you mention.
Bryan Costin
I think B would be even better if the layout were more like A. Needs more whitespace. Layout C is just unnecessarily complex. The process of deciding on a small purchase is sequential: Read what it is, find out how much, purchase. But two columns forces the user to juggle all three things as soon as the page loads. You’re asking to fill out a form to buy something before it’s even clear what you’re selling.
However, given some of the (apparently) misinformed comments I think all three designs have a more serious flaw: It’s not clear what the product is. First, let’s make it clear that I know nothing about this company except what’s in the screenshots, and I don’t care if they exist or not. But if I understand the copy right (esp the “Background” section seen on the first design) this “report” is a compilation of publically available information that law enforcement agencies must provide to anyone who ask, in accordance with child protection laws in the US. Some states do a decent job presenting the information online by themselves. Others do a bad job, thus creating a market for third-party sites like this.
Those may or may not be good or effective laws (take it up with your member of congress) but the result is certainly not some random “creepy guy” blacklist compiled by fly-by-night spammers as some have implied. Calling the author/designer unethical seems both uninformed and unfair.
Colin Fahrion
This article really missed the mark by not trying to really analyze why B was slightly better. As others mentioned here it probably had something to do with the testimonials being more avaiable and the huge “Click Here” steering the user past the jumble of content. Though personally I think the click here is just a cop out for a bad design with no clear path for the eye to follow. Sure C is confusing but so are the others—they just are better because at least there is a simple down direction to follow.
I agree with the article that real testing is great but really… how often can you get a client to pay for that? This article could have served better if it not only tested but analyzed the whys. For at least then we could walk away with a bit more understanding for use the next time a client won’t pay for design user testing.
Chris Pearson
This is sound advice for ANY business in ANY industry. Live testing is the only way to know what really works, and everyone should be open to the idea that release #1 may not be (and probably isn’t) the most effective version.
For the record, I used live testing in a phone card distribution business, and was able to come up with a protocol that allowed sales to increase linearly (to keep up with cash flow) in an almost unbounded fashion. The results after 18 months of this gameplan? 130% increase in sales, 400% jump in profits, 81% jump in per-unit profit.
When I read something like this, it just serves to drive home a point that I was lucky enough to have firsthand experience with. I hope everyone who reads this sees the value here.
Alex Walker
I understand the point the author is trying to make but the methodology is appalling and acts to counter, the useful point that I think is lurking in the article.
To make a test meaningful you can only alter one variable. How can you tell what caused the difference in sales if both copy and layout was changed. As an earlier commenter suggested, perhaps it was the copy that had the real effect, not the layout as the author implies.
Jon Smith
The major difference to the results was copy changes, not design. The authow even stated as much in the opening: “Version B follows the same basic layout, but we made some minor copy changes.”
Maak Bow
To my eyes all 3 options looked confusing….didnt know where to look or what different area’s did.
Hire a professional web designer to actually design the site rather than tinkering with slight variations…. THEN see the results.
aliyu aliyu
Its very difficult sometimes to determine what kind of layout to use for a particular site.
Shashank Sondhi
When I clicked the Marketing Experiments’ Journal link in our author’s profile it took me to alovely white background page with one sentence on it… No website is configured for this URL.
Why is that happening…?
adjava john
I have a great website and i want to add articles source in this site, plz help me. I want articles on trade and business.
Damian Stephens
The author’s condescending remark, “I know they won’t win any design awards. But they are functional and familiar” betrays the offensive implication that designers simply decorate web sites with pretty graphics, with no understanding of what they are doing.
The article is not ‘provocative’ because it is suggesting something radical or novel, it is provactive because of it’s crass arrogance and ignorance.
Surely in 2006 we’re not still debating the value of design! The purpose of design (whether in print or web) is to aid the communication of ideas and information, and to create desire.
I would be amazed if an IA, information designer or graphic designer had been anywhere near the example pages that were shown. All three were terrible – it’s a wonder that site made any sales at all!
I am all for live testing (under rigorous conditions),,, but does the author of this article (which I have to say, has no place on this illustrious site) seriously believe that as designers we are just poking around in the dark hoping for the best?!
P R
First: Back to the basic purpose of the article: improving website design. Still pertinent and still needing to be examined…even in this lofty year of 2006! In fact, what seems to be missing in relating to the comparision test comments—is the realization of what is the difference in the effects between A, B, C.
Too many of the comments are concerned with details. Without a doubt, the best version was site B. Why? Structurally it was ergonimically appreciated visually (meaning eyeballroll) and did not ‘yank’ the head down as was version A’s affect…which to a tired neck, is a complaint. Further, a minor frustration that the core of the info which immediately ‘weights’ seems to be sinking into the bottom of the computer screen.
Test it yourself. Where are your eyes immediately drawn to before you reavert to search the site? Looking right down at the bottom of the screen. Now you understand the effect of the neck being ‘yanked’.
In version B, the bulk that the eye first seeks is better positioned to the upper left, and is more effortless to view overall.
clay davis
forgive me if this was mentioned already but I think b was most succesful simply because of the “want names pictures and address” link near the top. who wouldn’t avoid mucking through such a bad layout if they could click on a link and get what they really wanted? the rest of the design choices in my opinion are irrelevent.