Quick and Dirty Remote User Testing

by Nate Bolt

19 Reader Comments

Back to the Article
  1. Thanks for the ideas.

    “Skype”:http://www.skype.com/ has screen sharing now too. (There is a bug where the cursor blinks for some video cards, but I figure they will have that fixed soon.)

    Copy & paste the code below to embed this comment.
  2. Very informative article,
    I would go with the third option.

    Thanks

    Copy & paste the code below to embed this comment.
  3. Also consider Open Web Analytics (http://www.openwebanalytics.com) if you are looking for a free open source alternative for user mouse recordings and click heatmaps.

    Copy & paste the code below to embed this comment.
  4. Great writeup on user testing – Silverback has been my favorite, but now I have a whole slew of other applications to try out. Thanks (and thanks for the link as well!).

    Copy & paste the code below to embed this comment.
  5. This is a really great breakdown of a lot of different tools – sometimes it can get confusing out there with so many of them floating about, but as you rightly point out, in many cases they can serve a different purpose.

    Personally I like the tools with a lower time commitment, but of course it depends entirely on the projects you are working on, and what kind of feedback you need to solicit.

    I completely agree with your opinion about the paid panel kind of apps; I’ve always worried if perhaps those kind of panels aren’t a little self selecting.

    The good thing (in my opinion) about tools like ours (I’m from http://IntuitionHQ.com) is that you can just send a simple 2 minute tests off to the sites stakeholders and get almost instantaneous feedback about which parts of the site are or aren’t working well, and then proceed to a newer, more usable revision.

    Once again, thanks for sharing, very enjoyable.

    Copy & paste the code below to embed this comment.
  6. Hiya Nate.
    Thanks for your overview. .. also nice to see another kiwi (intuition HQ) here reading your post.
    I’ve recently run some sessions using Skype and wrote up a post to share my experience.
    http://www.userexperience.co.nz/2010/05/skype-takes-the-hassle-out-of-remote-usability/
    All the best … Nick

    Copy & paste the code below to embed this comment.
  7. Nate,

    I enjoyed the article, thank you!

    One more tool to add to Method #3—“Concept Feedback”:http://www.conceptfeedback.com is a design review community where websites (or other designs) can be posted for free (after giving 5 reviews) and receive feedback from a community of 6,000+ designers and developers.  Reviewers are not paid, they just choose to do it for the betterment of design I guess :)

    Andrew

    Copy & paste the code below to embed this comment.
  8. So glad you guys enjoyed the article! Couple issues from the comments so far:

    • Skype **
      We use Skype damn near religiously for our research calls, but find that actually calling people on Skype is more of a pain when you are recruiting from a general population. Everyone has a phone and a web browser, but not everyone has Skype running and is willing to share their Skype ID (at least for now)
    • Other tools **
      Thanks guys for posting your various other tools – Concept Feedback I hadn’t seen yet! I will add to http://remoteusability.com/tools

Copy & paste the code below to embed this comment.
  • Hi,

    Great article. We use Netviewer Meet a lot (netviewer.com). It is designed to give presentatons online, but you can hand over the desktop and take a peek at what the user is doing. We use a regular phone for voice. The VOIP in Netviewer is crap.

    What is good about Netviewer is that it is easy to install for the user (actualy it is not installed sort of). And it works all the time, even behind the most bad ass firewalls.

    Maybe it can wordk for you.

    Cheers.

    Copy & paste the code below to embed this comment.
  • The best think about this one is a HUGE amount of useful links. Thanks.

    Copy & paste the code below to embed this comment.
  • I am just starting to get into user testing and hadn’t done it much before in the past with any of my designs, being a student, but am finding it to be a lot more helpful now.  This article was great! A good tool to use when testing, also, is a program called Denote – it basically lets you or the client make notes on the website, which you or whoever has access can view and then comment on or “check off” once you’ve addressed the note.  I found it here: www.denoteapp.com.  Anything to make life a little easier!

    Copy & paste the code below to embed this comment.
  • I really appreciate this article because it spells out for me how user testing is approached from three different methods.  Sometimes, depending on the site and service you are designing for, it makes more sense to conduct user testing through real live people, whereas, in other cases, heat map software is preferable.  Being able to select and combine strategies and methods of user testing is a practice i would recommend.

    Copy & paste the code below to embed this comment.
  • Hi,

    thanks for another great article. I would recommend that you try this tool: “Mouseflow.com”:http://mouseflow.com

    You get the most cost effective remote usability studies. Watch videos of your visitors, analyse pages with heatmaps and scroll-maps, watch a live-stream of your current visiting users. Free + paid plans.

    Copy & paste the code below to embed this comment.
  • Thanks for the article, I’m pretty interested in usability and as i start to work on bigger projects it;s becoming more and more important (the small projects are also v important, of course).

    I wish the designer at KFC had read this article, they have a new self service checkout here in Leicester (UK), some of the menu options are very confusing and it seem to have some poor overall usability. I’m sure those guys could afford some cheap and dirty user testing!

    Copy & paste the code below to embed this comment.
  • “Don’t interrupt them in the middle of tasks, even if they seem to be running into trouble or “going off-track”—it’s important to see where they get stuck. Ask open-ended, non-leading questions such as, “What’s going through your mind right now?” or “Can you tell me what you’re looking at now?”

    > So that’s all you need then?  Excellent!  A couple of instructions and some sort of *tool”, and you’re a usability engineer!

    > No need to worry about designing tasks, recruiting the right people – any real kind of prep.  And facilitation means only two things to remember.  How convenient.  Analysis?  Not much to that either, right?  Hardly worth a mention.

    > Remember desktop publishing?  All of a sudden, anybody could create a brochure or newsletter.  And it showed.  Desktop publishing did not convert Aunt Edna or the guy in HR into a graphic designer.  And these usabilty tools are not going to turn graphic designers and deveopers into usability engineers.

    Copy & paste the code below to embed this comment.
  • Great article Nate. I believe that virtual classrooms work great for remote testing too. Have you tried “WizIQ”:http://www.wiziq.com ? Although they say their virtual classroom is suitable for teaching and learning, I think it’s good for any synchronous communication. Thoughts anyone?

    Copy & paste the code below to embed this comment.
  • Quality structured user testing is something that we have certainly overlooked in the past. We used to rely mostly on general feedback from clients and people we know who were willing to have a quick look at what we’d developed.

    We have used Forrst (http://forrst.com/) for design and dev feedback, which is similar to Concept Feedback.

    This article has inspired us to employ some of the suggested techniques and tools to improve the way we develop sites in the future. Should make testing a lot more rewarding.

    Copy & paste the code below to embed this comment.
  • Thanks for a really interesting article.  As someone who has been working (off and on) in UX for a good few years, it’s great that people are coming around to the idea of actually making a point of asking users what they think. 

    Seems obvious but in my experience there are loads of companies who still don’t really get it.  I guess part of the job of UX professionals is now to educate clients on the benefits of effective user testing.

    To do my bit, I created a remote user testing service called Kupima (https://kupima.com) which I hope offers a compelling alternative to some of the more established services.  I’d be delighted to hear from anyone who might want to give it a try or review it.  Of course, it goes without saying that it’s far from the only choice out there, and the most important thing I feel is that people do something (whether it’s with Kupima, a competitor or simply one of the other techniques discussed). 

    The insight you can get from actually engaging with users makes it all worthwhile – and it’s a wonder that more people haven’t caught on to this idea.

    Copy & paste the code below to embed this comment.
  • Hi Nate, great summary, mostly agree (as usual…:)), except for two things:

    1. I disagree with your statement that facial expressions are not important in usability/user experience testing/design (you said “you have to trust me on this…”).
    Having done thousands of unattended or unmoderated usability tests, I can attest to the fact that recording the respondent’s expression provides about 55% more information than otherwise, on average (who they are, what kind of socioeconomic background are they from, what mood are they in, are they alone or not, are they distracted, how long before a moment of frustration translates into a verbalized thought, how often frustration leads to abandonment with no comment, etc)

    2. You mention that a number of platforms, including ours, userlytics, largely depend on an army of paid testers.
    I cannot speak on the other platforms you mention, but this statement is inaccurate in our case; almost all of our respondents are recruited by us on a customized basis,  and often are highly specialized B2B low incidence populations where the type of paid testers you refer to would not work well.

    In fact, this is an area where including a webcam view of the participant adds a lot of value; you can see and verify that your tester/respondent is truly your target Persona.

    That said, we do offer crowd-sourced testers where clients do not have a large budget, as well as screen recording + audio without webcam views, for the same types of situations, and for many situations, that is good enough.

    Copy & paste the code below to embed this comment.