Comments on Beyond Usability Testing

2 Reader Comments

Back to the Article
  1. I like your article Devan and I’m pleased you’ve pointed people to the list of guidelines we use for expert reviews at Userfocus—but I don’t understand why you seem to couch this as an either-or discussion. Any sensible project manager will include time for both usability testing (the gold standard) and for expert reviews. I’m sure it’s not your intention, but people may read your article and think that expert reviews are a cheap alternative to user testing. Expert reviews are best used to achieve an extra iteration through the design loop, not as a replacement for user testing.

    Copy & paste the code below to embed this comment.
  2. Thanks for the comment, David (and for the excellent set of guidelines). It’s good to know that the article seemed to you to be an either-or, because that certainly wasn’t my intention, and I’m glad to have the chance to discuss the question further.

    So, let me say: You and I agree, I think, that there’s no substitute for testing with actual or potential users, that every effort should be made to involve real users in every project, and that expert review and heuristic evaluation work best when they multiply the value of a usability study rather than supplanting the study itself. I may not have been emphatic enough on these points, largely moving on from them after my first two paragraphs.

    At the same time, I do see certain circumstances where involving real users just won’t work as well as one of these other measures. Perhaps a project has run out of money or is running behind against a hard launch deadline. Ten hours of an expert’s time will be cheaper and can be put into action much more quickly and effectively than even a guerilla usability study, in my experience. (Some projects also have no budget at all, as with some non-profit clients I’ve had. Personally, I’m more willing to do expert reviews _pro bono_ than I am usability testing.)

    I’d also say that it’s common for usability to be a last priority. If my clients or their bosses are the wrong combination of unwilling to pay for a full-scale usability study and skeptical that guerilla studies have any value—both positions I find misguided and try to help work against—they are often nonetheless willing to enlist an expert to spend a week reviewing a site. Such a review not only improves a site or design, but in some case helps make the case for later usability testing, since many reports, if they’re honest, will have to identify some points on which there will be, again, no substitute for the input of real users.

    None of these situations is ideal, of course. I wish every project brought real users into the process. On my view, though, expert reviews and heuristic analyses can be stop-gap measures in a world where usability testing just isn’t conducted as much as we’d like. While we’re fighting the fight for more of it, though, I think we might as well also help the users with less direct methods on the table here.

    Copy & paste the code below to embed this comment.
  3. Sorry, commenting is closed on this article.