Thanks for the comment, David (and for the excellent set of guidelines). It’s good to know that the article seemed to you to be an either-or, because that certainly wasn’t my intention, and I’m glad to have the chance to discuss the question further.
So, let me say: You and I agree, I think, that there’s no substitute for testing with actual or potential users, that every effort should be made to involve real users in every project, and that expert review and heuristic evaluation work best when they multiply the value of a usability study rather than supplanting the study itself. I may not have been emphatic enough on these points, largely moving on from them after my first two paragraphs.
At the same time, I do see certain circumstances where involving real users just won’t work as well as one of these other measures. Perhaps a project has run out of money or is running behind against a hard launch deadline. Ten hours of an expert’s time will be cheaper and can be put into action much more quickly and effectively than even a guerilla usability study, in my experience. (Some projects also have no budget at all, as with some non-profit clients I’ve had. Personally, I’m more willing to do expert reviews _pro bono_ than I am usability testing.)
I’d also say that it’s common for usability to be a last priority. If my clients or their bosses are the wrong combination of unwilling to pay for a full-scale usability study and skeptical that guerilla studies have any value—both positions I find misguided and try to help work against—they are often nonetheless willing to enlist an expert to spend a week reviewing a site. Such a review not only improves a site or design, but in some case helps make the case for later usability testing, since many reports, if they’re honest, will have to identify some points on which there will be, again, no substitute for the input of real users.
None of these situations is ideal, of course. I wish every project brought real users into the process. On my view, though, expert reviews and heuristic analyses can be stop-gap measures in a world where usability testing just isn’t conducted as much as we’d like. While we’re fighting the fight for more of it, though, I think we might as well also help the users with less direct methods on the table here.