It’s not breaking news to say that the core of UX, in a vacuum, is talking to your users to gather insights and then applying that information to your designs. But it’s equally true that UX does not happen in a vacuum. So what happens when you don’t have the budget or the timeline to run user tests, card sorts, or stakeholder interviews? What should you do when your company doesn’t want you bothering the paying customers who use their software? In short, how do you do UX research when you can’t get direct access to your users?
While the best methods for gathering user insights entail first-hand research, there are other ways to quickly glean qualitative data about your users’ wants and needs—beyond the usual lightweight guerrilla user testing options.
For a start, companies that are new or have a smaller digital footprint can benefit from things like forums or even competitor reviews to get a better sense of the users in their industry vertical. And for more established companies, customer service logs and app reviews can be invaluable for learning what users think about specific products. Let’s check out a few techniques I like to recommend.
App reviews#section2
When products have a mobile app component, I start looking at reviews posted on the App Store or Google Play. The key, in terms of user research, is to focus on the substance of what the user is saying, as opposed to the rating (i.e., one star to five stars). For instance:
- Is the user simply disgruntled or are they asking for a tangible feature to be added to the product?
- Is the user truly thrilled with some aspect of her experience using the app or is she just a brand loyalist?
While reviews do tend to be rather partisan, keep in mind that users are most likely to leave feedback when motivated by an emotional reaction to the product. Emotionally-driven reviews—whether positive or negative—tend to be outliers on the bell curve, so the next step is taking all those reviews and distilling them into tangible insights. Let’s face it, when you want to improve the featureset and functionality, a general reaction to the entirety of an app doesn’t tend to be particularly actionable. Here are a few questions I always start with:
- Are there missing features users want to see?
- Do users seem confused by aspects of the UI?
- Are they complaining about bugs or performance issues that are popping up and making the app unusable?
- Do people really love a hidden feature that was put in as an afterthought with minimal prominence—something we should consider placing more front and center?
- Does it seem like people understand how to use the app or do they need a tutorial on first open?
Also, it’s important to remember that feedback on an iOS app may or may not be applicable to an Android app (or mobile web experience), and vice versa.
Customer service logs#section3
Customer service and help center personnel are on the front lines with your users, helping them with specific struggles they encounter with the usability of your products. In other words, they’re constantly learning how users see the product and go about using it.
Since user information can be sensitive, the first thing to try is asking whether service calls and contacts are being logged. If so, ask whether it’s possible to get access to the records for user research purposes. If there are no logs, or if you are unable to get access to them, see if a few brief stakeholder interviews with customer service team members is an option. Use the interviews to learn which types of problems and complaints they routinely field.
Given the nature of customer service and the purpose of help centers, it’s likely that much of the feedback will be negative. Even so, these logs can still provide excellent data. In particular, the feedback can help illuminate policies and business practices that are creating a negative user experience, not to mention identify the points at which they occur during the user journey. And remember, your user experience is about more than just the design of your app and website.
Contact form emails#section4
“Contact Us” forms and messages can be rich with direct input from your users. Obviously, the first things to look for are complaints about an aspect of the site itself. For example, are users struggling to find a feature or getting confused by a certain page on the site? Beyond that, the forms themselves can relate to aspects of the user journey that are problematic.
If a brand or company does not have this feature for gathering site visitors’ opinions, it’s relatively easy to add a contact form, in terms of development effort. However, it’s important to note that if you have a contact form on your site, someone should be actively monitoring it and responding to users.
Industry forums#section5
While the likes of Reddit and 4Chan have given the world of online forums some questionable connotations, the truth is that many online forums are also excellent sources of information about how digital products are operating in the wild, and how specific products and trends as a whole are influencing users. The research insights might be less obvious, but they’re easy to spot if you’re looking for them.
A look at the Apple TV Apps section of Mac Rumors reveals that many users of the 4th generation Apple TV have a problem with the YouTube app not fading out video titles when a video is playing. Similarly, a brief review of the Delta Airlines thread on FlyerTalk shows that users have questions about everything from Economy Plus seating to the Delta and American Express credit card. Reviewing this information could help Delta retool the content strategy and information architecture of their mobile app to address questions more clearly.
Many forums are industry specific, and therefore not applicable to every situation. There are just as many out there, however, that specialize in spanning numerous industries. Ars Technica covers virtually any sort of traditional tech product. For video games, IGN offers helpful feedback from players about everything from game length and storyline to controls. For nutrition and exercise, Bodybuilding.com’s BodySpace forum is a top online destination for users to discuss nutrition and exercise. Of course, not every forum offers in-depth discussions regarding specific apps, websites, or even companies, but each provides great sources of information about what motivates and interests consumers in that industry vertical.
Multi-topic forums can be searched for company- or product-specific threads. Reddit (despite its aforementioned reputation) features thriving, engaged communities of actual users talking about topics of value. Quora offers an almost scholarly approach to the format, with many users possessing strong subject matter credentials to validate their expertise.
Reviews of competitors#section6
Perhaps your brand or product is new in the market and there’s not yet enough data from any of these sources to be actionable. So what then? Find out what your potential users have to say about the competition. If you want to launch a car service, see what users say about Lyft and Uber on the App Store. Want to improve Jet? Read reviews of Amazon Prime. Do you work for InstaCart? Find out what users have to say about Fresh Direct.
In summary#section7
There’s still no substitute for actually talking to your user base, whether that’s initial research in the form of stakeholder interviews or testing design iterations, but even when that’s not an option, there’s no excuse for not gathering feedback from your users.
Good UX design should always be based on user insights, not assumptions about best practices or what might translate from other products and industries. So go find out what your users are saying. From Yelp! to Glassdoor to App Store Reviews, consumers are readily sharing their opinions about businesses of every size, in every industry.
I was in the same situation, working on an ipad-based Assistive and Augmentative Communication app, without access to users. I scoured app reviews of similar products, which not only helped me gain insight into valued features, but even to create rough personas because parent’s reviewing the apps quite often went into detail on their particular child’s abilities, deficits, and reactions to other apps they tried.
That’s a great example Chris! The unfortunately reality of UX is that more often than not you don’t have the time/budget/access you need to really do robust user research. So being willing to roll up your sleeves and scrape what data you can is often the difference between good UX and bad UX. Even having just a few data points gives you something to properly prioritize content and features. Otherwise, it’s far too easy to jump right into concepting and fall in love with a feature set or design, and then that actually starts to drive the way you look at data down the road. That in turn, ultimately leads to your tactics driving the strategy instead of vice versa.
I used to work on landing pages with contact forms. We got the back end developer to set up a little system to track form engagement. Basically, for each user, we tracked which form fields they filled out. Not the values, or any identifiable information–just whether or not they entered a value. If they filled out the form, we discarded the data. We only held onto it if the user abandoned the form. We soon found out what form fields users were most apprehensive about filling out and how field placement affected conversion rate. Great info, and we didn’t have to bother the users at all.
Great article Jon, though you’re missing one important fact. With tools out there today you can create an entire feedback cycle with your users in minutes and for free. We use gotcha.io for all of our MVPs
Great article Jon.
Excellent article and one we will be sharing with our UX students. Another method is to go into specific groups in Facebook and (if the group allows) post a poll. This has been very useful for us as we are able to then look at the profile and validate they are target persona.
Great examples from Brandon and Web Courses – at the core of handling a situation where you can’t do traditional UX research is to be innovative and work with what you have available. There are certainly more methods available than what I listed, and these are all great examples.
App reviews are highly overrated specially when anyone can spam as well as submit fake reviews.
Contact forms and customer service logs are more reliable compared to app reviews.
My students did a homework about this and one of the best idea they got is to use “promotional codes” with prizes, then many consumers (any type of them) will surely want to answer the online survey.
But the differents ideas you wrote are very interesting too. Good job.
Good article. I would only add one thing – if you can’t talk directly to your users, talk with people who can. In the past, when working for e-commerce company, I have interviewed delivery van drivers, who had direct contact with customers. Its not ideal, but you can get a bit more insightful data than log analysis 🙂
Best,
Very Inspirational
I’ve had great results using demographic surveys and google analytics to create personas. I write the surveys in HTML and distribute them with Amazon Mechanical Turk.
great job
great article! a very cheap and instant way to check usability of an app or a website in the target group is by doing Unmoderated Remote Usability Testing online. Platforms like http://www.rocketmonkeys.io offer this service.
very intersting topic admin thanks for this
Good one. As technology is getting advanced everyday, and so are the users thoughts. I believe in giving my customers the best UX design as I can. Its important to think from their point of view. For affordable Web designing and development please contact me.
Great ideas here. I’d like to add one.
When we are doing user discovery, and don’t have access to actual users, we like to include what we call a “user proxy” in the room. This could be a customer service rep, a product person, etc. It just needs to be someone who understands the customer / user, and who also understands the product well. They can speak on behalf of their user better than anyone else on the team, and are the next best option to getting the users themselves in the room.