I am just starting to get into user testing and hadn’t done it much before in the past with any of my designs, being a student, but am finding it to be a lot more helpful now. This article was great! A good tool to use when testing, also, is a program called Denote – it basically lets you or the client make notes on the website, which you or whoever has access can view and then comment on or “check off” once you’ve addressed the note. I found it here: www.denoteapp.com. Anything to make life a little easier!
I really appreciate this article because it spells out for me how user testing is approached from three different methods. Sometimes, depending on the site and service you are designing for, it makes more sense to conduct user testing through real live people, whereas, in other cases, heat map software is preferable. Being able to select and combine strategies and methods of user testing is a practice i would recommend.
thanks for another great article. I would recommend that you try this tool: “Mouseflow.com”:http://mouseflow.com
You get the most cost effective remote usability studies. Watch videos of your visitors, analyse pages with heatmaps and scroll-maps, watch a live-stream of your current visiting users. Free + paid plans.
Thanks for the article, I’m pretty interested in usability and as i start to work on bigger projects it;s becoming more and more important (the small projects are also v important, of course).
I wish the designer at KFC had read this article, they have a new self service checkout here in Leicester (UK), some of the menu options are very confusing and it seem to have some poor overall usability. I’m sure those guys could afford some cheap and dirty user testing!
Copy & paste the code below to embed this comment.
cliff a
“Don’t interrupt them in the middle of tasks, even if they seem to be running into trouble or “going off-track”—it’s important to see where they get stuck. Ask open-ended, non-leading questions such as, “What’s going through your mind right now?” or “Can you tell me what you’re looking at now?”
> So that’s all you need then? Excellent! A couple of instructions and some sort of *tool”, and you’re a usability engineer!
> No need to worry about designing tasks, recruiting the right people – any real kind of prep. And facilitation means only two things to remember. How convenient. Analysis? Not much to that either, right? Hardly worth a mention.
> Remember desktop publishing? All of a sudden, anybody could create a brochure or newsletter. And it showed. Desktop publishing did not convert Aunt Edna or the guy in HR into a graphic designer. And these usabilty tools are not going to turn graphic designers and deveopers into usability engineers.
Copy & paste the code below to embed this comment.
Peteford
Great article Nate. I believe that virtual classrooms work great for remote testing too. Have you tried “WizIQ”:http://www.wiziq.com ? Although they say their virtual classroom is suitable for teaching and learning, I think it’s good for any synchronous communication. Thoughts anyone?
Quality structured user testing is something that we have certainly overlooked in the past. We used to rely mostly on general feedback from clients and people we know who were willing to have a quick look at what we’d developed.
We have used Forrst (http://forrst.com/) for design and dev feedback, which is similar to Concept Feedback.
This article has inspired us to employ some of the suggested techniques and tools to improve the way we develop sites in the future. Should make testing a lot more rewarding.
Thanks for a really interesting article. As someone who has been working (off and on) in UX for a good few years, it’s great that people are coming around to the idea of actually making a point of asking users what they think.
Seems obvious but in my experience there are loads of companies who still don’t really get it. I guess part of the job of UX professionals is now to educate clients on the benefits of effective user testing.
To do my bit, I created a remote user testing service called Kupima (https://kupima.com) which I hope offers a compelling alternative to some of the more established services. I’d be delighted to hear from anyone who might want to give it a try or review it. Of course, it goes without saying that it’s far from the only choice out there, and the most important thing I feel is that people do something (whether it’s with Kupima, a competitor or simply one of the other techniques discussed).
The insight you can get from actually engaging with users makes it all worthwhile – and it’s a wonder that more people haven’t caught on to this idea.
Hi Nate, great summary, mostly agree (as usual…:)), except for two things:
1. I disagree with your statement that facial expressions are not important in usability/user experience testing/design (you said “you have to trust me on this…”).
Having done thousands of unattended or unmoderated usability tests, I can attest to the fact that recording the respondent’s expression provides about 55% more information than otherwise, on average (who they are, what kind of socioeconomic background are they from, what mood are they in, are they alone or not, are they distracted, how long before a moment of frustration translates into a verbalized thought, how often frustration leads to abandonment with no comment, etc)
2. You mention that a number of platforms, including ours, userlytics, largely depend on an army of paid testers.
I cannot speak on the other platforms you mention, but this statement is inaccurate in our case; almost all of our respondents are recruited by us on a customized basis, and often are highly specialized B2B low incidence populations where the type of paid testers you refer to would not work well.
In fact, this is an area where including a webcam view of the participant adds a lot of value; you can see and verify that your tester/respondent is truly your target Persona.
That said, we do offer crowd-sourced testers where clients do not have a large budget, as well as screen recording + audio without webcam views, for the same types of situations, and for many situations, that is good enough.
19 Reader Comments
Back to the Articleamicablyamy
I am just starting to get into user testing and hadn’t done it much before in the past with any of my designs, being a student, but am finding it to be a lot more helpful now. This article was great! A good tool to use when testing, also, is a program called Denote – it basically lets you or the client make notes on the website, which you or whoever has access can view and then comment on or “check off” once you’ve addressed the note. I found it here: www.denoteapp.com. Anything to make life a little easier!
UI Web Design
I really appreciate this article because it spells out for me how user testing is approached from three different methods. Sometimes, depending on the site and service you are designing for, it makes more sense to conduct user testing through real live people, whereas, in other cases, heat map software is preferable. Being able to select and combine strategies and methods of user testing is a practice i would recommend.
mouseflow
Hi,
thanks for another great article. I would recommend that you try this tool: “Mouseflow.com”:http://mouseflow.com
You get the most cost effective remote usability studies. Watch videos of your visitors, analyse pages with heatmaps and scroll-maps, watch a live-stream of your current visiting users. Free + paid plans.
David-Web-Designer
Thanks for the article, I’m pretty interested in usability and as i start to work on bigger projects it;s becoming more and more important (the small projects are also v important, of course).
I wish the designer at KFC had read this article, they have a new self service checkout here in Leicester (UK), some of the menu options are very confusing and it seem to have some poor overall usability. I’m sure those guys could afford some cheap and dirty user testing!
cliff a
“Don’t interrupt them in the middle of tasks, even if they seem to be running into trouble or “going off-track”—it’s important to see where they get stuck. Ask open-ended, non-leading questions such as, “What’s going through your mind right now?” or “Can you tell me what you’re looking at now?”
> So that’s all you need then? Excellent! A couple of instructions and some sort of *tool”, and you’re a usability engineer!
> No need to worry about designing tasks, recruiting the right people – any real kind of prep. And facilitation means only two things to remember. How convenient. Analysis? Not much to that either, right? Hardly worth a mention.
> Remember desktop publishing? All of a sudden, anybody could create a brochure or newsletter. And it showed. Desktop publishing did not convert Aunt Edna or the guy in HR into a graphic designer. And these usabilty tools are not going to turn graphic designers and deveopers into usability engineers.
Peteford
Great article Nate. I believe that virtual classrooms work great for remote testing too. Have you tried “WizIQ”:http://www.wiziq.com ? Although they say their virtual classroom is suitable for teaching and learning, I think it’s good for any synchronous communication. Thoughts anyone?
stunisbett
Quality structured user testing is something that we have certainly overlooked in the past. We used to rely mostly on general feedback from clients and people we know who were willing to have a quick look at what we’d developed.
We have used Forrst (http://forrst.com/) for design and dev feedback, which is similar to Concept Feedback.
This article has inspired us to employ some of the suggested techniques and tools to improve the way we develop sites in the future. Should make testing a lot more rewarding.
jtcedinburgh
Thanks for a really interesting article. As someone who has been working (off and on) in UX for a good few years, it’s great that people are coming around to the idea of actually making a point of asking users what they think.
Seems obvious but in my experience there are loads of companies who still don’t really get it. I guess part of the job of UX professionals is now to educate clients on the benefits of effective user testing.
To do my bit, I created a remote user testing service called Kupima (https://kupima.com) which I hope offers a compelling alternative to some of the more established services. I’d be delighted to hear from anyone who might want to give it a try or review it. Of course, it goes without saying that it’s far from the only choice out there, and the most important thing I feel is that people do something (whether it’s with Kupima, a competitor or simply one of the other techniques discussed).
The insight you can get from actually engaging with users makes it all worthwhile – and it’s a wonder that more people haven’t caught on to this idea.
arivas
Hi Nate, great summary, mostly agree (as usual…:)), except for two things:
1. I disagree with your statement that facial expressions are not important in usability/user experience testing/design (you said “you have to trust me on this…”).
Having done thousands of unattended or unmoderated usability tests, I can attest to the fact that recording the respondent’s expression provides about 55% more information than otherwise, on average (who they are, what kind of socioeconomic background are they from, what mood are they in, are they alone or not, are they distracted, how long before a moment of frustration translates into a verbalized thought, how often frustration leads to abandonment with no comment, etc)
2. You mention that a number of platforms, including ours, userlytics, largely depend on an army of paid testers.
I cannot speak on the other platforms you mention, but this statement is inaccurate in our case; almost all of our respondents are recruited by us on a customized basis, and often are highly specialized B2B low incidence populations where the type of paid testers you refer to would not work well.
In fact, this is an area where including a webcam view of the participant adds a lot of value; you can see and verify that your tester/respondent is truly your target Persona.
That said, we do offer crowd-sourced testers where clients do not have a large budget, as well as screen recording + audio without webcam views, for the same types of situations, and for many situations, that is good enough.