Comments on Usability Testing Demystified

23 Reader Comments

Back to the Article
  1. Maybe Jakob Nielsen should allow a designer to style, since this is the site I most often hear referenced by people who associate good usability with poor, outdated design

    Copy & paste the code below to embed this comment.
  2. Great article, Dana. Thanks for clarifying the value, purpose, and flavors of usability testing. From what I’ve seen, the bad rap seems to stem from teams working with less experienced practitioners who ask the wrong questions, leading questions, or believe they must use the formal lab approach vs. doing quick or in-context testing. Hope this helps squelch the naysayers.

    Copy & paste the code below to embed this comment.
  3. Thanks, Dana.  Very good information.  I’m always hearing that some testing is better than none at all. That being said, what do you think about  I’m considering it as an option for quick and inexpensive testing for some of our firm’s clients.

    Copy & paste the code below to embed this comment.
  4. @ronansprake - there was a project to redesign one of Nielsen’s pages, you can see the results at “”: . Nielsen does not pretend that his website is an example of good aesthetic design, he knows that it is not, but says that he does not have the skills to make it aesthetically pleasing and so he would rather leave it plain and simple than make it a mess.

    Despite its looks, it is one of the most popular websites out there for web authoring and usability research, which just proves his point - if you build it well, they will come ... even if it looks cruddy.

    Copy & paste the code below to embed this comment.
  5. I’ve only done one session, but I found people didn’t like being asked to “test” something - they didn’t want to take a test. I learned a lot that time, mainly about what I should and shouldn’t say.

    It’s difficult to work behind a screen everyday and then go into such a public and more than likely humiliating situation. Through humiliation comes humility though. I guess.

    Next time I will try the “gathering feedback” approach.

    Copy & paste the code below to embed this comment.
  6. Recently while talking to a “Usability Expert” at one of our local utility companies, she asked if I had any experience with different usability software programs that spit out usability data. I must have had a blank look on my face as I thought to myself, “how many software programs are going to be using your website?”

    Copy & paste the code below to embed this comment.
  7. bq. I’ve only done one session, but I found people didn’t like being asked to “test” something — they didn’t want to take a test. I learned a lot that time, mainly about what I should and shouldn’t say.

    What you need people to understand is that they aren’t being tested. They are the examiners, your website is under the microscope. If they are having difficulty, that means that the website probably isn’t well-enough designed to meet their needs. Because even if they are clearly not demonstrating a lot of intelligence, logic or resourcefulness - and there are a lot of people out there who don’t! - they are still the people that your website is there to serve, and you’ve got a better chance of changing your website to fit what people can work with than you have of changing the way they work with your website.

    Yes, sometimes it’s hard. When you’ve created a design that you think is really easy to use, and some knucklehead comes along and can’t follow simple instructions, ignores the big highlighted links and buttons that you’ve put in prime spots and clicks around randomly for a while before giving up, it’s hard not to scream and cry and ask them why they can’t see the blindingly obvious when it’s right in front of them ... but you need to be able to understand their thought processes and what they are looking for/at, so that you can come up with a design that they can follow.

    Copy & paste the code below to embed this comment.
  8. Many of us would like to do user testing, but how can resources which are typically thin be allocated? Resources meaning time and/or money. 

    Specifically, how can you gain meaningful input on something that s quite frankly not even to the “usable” stage in production yet, and getting it there would eat up some of the budget… only to learn from the testing that things have to be re-done? How is this reconciled?

    Copy & paste the code below to embed this comment.
  9. @Bill Leonard

    I come from a background in game design and have implemented some of the techniques that I learned from game development into my web development projects.

    I always create a design document upfront that not only describes in text exactly what the goal of the project is, but also provides well thought out visual diagrams of how the application will work. The diagrams do not have to be an exact replication of what the user-interface will look like. Think of it as the sketch that a painter lays on a canvas before actually painting.  The main goal is to be able to quickly create and modify a tangible representation of the process that the user will have to go through in order to achieve the desired goal.

    This design procedure provides the ability to immediately identify unnecessary actions that the user will have to go through to navigate your application.  You can also easily create several diagrams of different ways that the navigation could work.  You can then have a simple pre-test by presenting the diagrams to multiple people from your user demographic and getting a consensus on which UI seems to be the most user friendly.

    This will allow you to focus your development from the start before spending too much of your budget.  You should always do a usability test once you’ve got a working prototype, but a good design document goes a long way in making sure that the first prototype is not a waste of time, money or resources.

    Copy & paste the code below to embed this comment.
  10. The problem about testing the usability of early prototypes is that you have to find many more people to test it on. I realized that most of the people who tested early versions of a design do not want to spend time over again. Probably enterprises with a considerable budget do not have difficulties solving this problem, but for small projects good usability testing is challenging anyway.

    Copy & paste the code below to embed this comment.
  11. @ Dana Chisnell

    Thanks for providing the great resources.  The article on Boxes and Arrows is spot on.  Many of the projects that I have been working on recently have extremely tight budgets.  That being said, we are usually pinching pennies during development.  Treating my testers well has really paid off for me. 

    I’ve been able to establish a good relationship with 5-6 people who will participate in tests as often as I need them.  These users are excited to see how their feedback effects our applications and always willing to test the prototypes for no cost.  Usually, we will provide lunch or other amenities before or after the test. That is enough to keep them happy.

    These guys are great at finding bugs in our systems and declaring what changes they would like to see.  I always test on a larger group once my core group of testers seem comfortable with the app. This process saves us a good amount of time and money.

    Copy & paste the code below to embed this comment.
  12. Thanks for that article, very informative.
    Whenever I’m designing and considering usability, I always think of the classic example of the newly built student campus. Put the buildings in place, but not the footpaths. After a couple of months, the students will naturally choose the easiest path between buildings and wear a groove in the grass. Build the footpaths there!

    Copy & paste the code below to embed this comment.
  13. Interesting article.

    I’m a big believer in testing but sometimes I get lazy. I think the feedback idea rather than testing is a great idea.

    I think content is key but visuals are important

    Copy & paste the code below to embed this comment.
  14. Usability testing may be an arduous process and it could very well be a luxury for some websites, but no matter what, it will always provide some insight into your website and your users.

    It’s amazing to realize that you can spend weeks hammering out a quality sitemap, navigation structure, and labels for important links, only to find that the actual visitors you are targeting don’t use the same terminology you do, and therefore can’t find what they’re looking for (even if its right in front of them).

    Usability testing doesn’t have to be burdensome — hold a simply focus group and ask participants to complete a “homework assignment” you’ve come up with. In the end, you can see how many were successful and give them a survey to add in their thoughts and frustrations with the project.

    Copy & paste the code below to embed this comment.
  15. Thanks Dana, great article. Thought you might be interested in our new fast and inexpensive service.

    It can be used for any type of online property (websites, website prototypes, online adverts, search processes, etc).

    It has a very attractive delivery speed, and price point (24-48 hours from order, 299$ for a 5 person user experience test). The features are as follows:

    Clients define a target url (their own or that of their competition or best practice)
    Clients define a goal for the testers to perform (e.g.; “find product x and take it through the checkout process..”)
    Clients define the demographics of the kind of testers they would like
    Clients define survey questions

    Within 24-48 hours clients receive a report that includes, for a minimum of 5 testers:

    Web cam recordings of the testers conducting their assigned goal/task
    A synchronized recording of the entire screen session during the test
    ClickFlow Analysis
    Contextual written “bubble” commentary on screenshots
    Survey results
    Other quantitative data

    Visitors to the site ( can request a free 1 person sample test.

    Kind regards,


    Copy & paste the code below to embed this comment.
  16. We use this and it is as Dana says, about evaluation,testing and great design. I think that the method of Alpha to Beta to release is key to success in any web application. Usertesting benchmarks the experience and I think you are right not to be a fan of this. I am for the before launch and after launch approach which emplys actually user information, market research and post launch click heat mapping.

    Copy & paste the code below to embed this comment.
  17. If you are a small company and dont really have the resources for such a large scale usability test, would you just carry out in house usability testing for a new piece of software or invite a select number of your regular customers to test the software online and give you feedback. As an online company its kind of difficult to set up a testing environment where you can visually see how people react, you can only get written feedback via a form. Where I think asking the right questions will determine how good and useful that feedback will be.

    Copy & paste the code below to embed this comment.
  18. For those that are looking to get feedback on your websites, I would recommend trying  I do not own this site, however, I am one of the testers that review websites.  The site shows website owners how visitors (such as myself)view/would use their website through the power of video.  Site owners write out task that they want the testers to complete.  The testers complete the tasks and send their videos to the site owner.

    Copy & paste the code below to embed this comment.
  19. Sorry, commenting is closed on this article.