Validating Product Ideas

Amazingly, 198 out of the 200 enterprise product managers and startup founders interviewed for this book said they were keeping a list of product ideas they wanted to make a reality some day. While keeping a wish list of solutions is a great thing to have, even more impressive is what only two startup founders were doing. These founders were keeping a list of problems they wanted to solve. They chose to first fall in love with a problem rather than a solution.

Focusing on learning how people solve a problem as IDEO did for Bank of America can lead to innovative solutions, or in this specific case, a successful service offering. IDEO designers and Bank of America employees observed people in Atlanta, Baltimore, and San Francisco. They discovered that many people in both the bank’s audience and the general public often rounded up their financial transactions for speed and convenience. They also discovered that moms were not able to save money due to a lack of resources or willpower. The result married these two observations into “Keep the Change,” a Bank of America checking account (Figure 3.1). This account “rounds up purchases made with a Bank of America Visa debit card to the nearest dollar and transfers the difference from individuals’ checking accounts into their savings accounts.” In less than a year, the offering attracted 2.5 million customers, generating more than 700,000 new checking accounts and one million new savings accounts for Bank of America.

Why Is This Question Important?#section2

The question is important because it’s the biggest blind spot (Figure 3.2) of the Lean Startup approach and its Build-Measure-Learn feedback loop concept. Getting feedback on a product and iterating is generally a convergent process based on what you know, what you experience in the world, what your friends tell you, what you whiteboard with your team, and what analytics tell you about your product. But sometimes the solution is outside.

The question “How do people currently solve a problem?” is critical, because deeply understanding a problem can go a long way toward solving it with a product, feature, or service. Falling in love with a problem happens through observing it happen in a relevant context, where the problem is occurring to people in your target audience.

The modern GPS network is a great example of how identifying this blind spot resulted in a solution.¹ The GPS network was originally built in the 1970s for the U.S. Navy and Air Force as a tracking system for planes, boats, and missiles. The Army, however, had always had a problem with mobile ground forces losing their way during a battle. They needed a reliable tracking mechanism for ground navigation. Obviously, the army’s need came from real, life-threatening situations where fighting units found themselves in the wrong place at the wrong time or late to arrive to the right place due to mistakes in manual navigation. The personal navigation systems and apps developed for this need are now what we use today as GPS devices on our smartphones.

When Should You Ask the Question?#section3

All. The. Time. Assuming you and your team have fallen in love with a problem to solve, constantly asking (and answering) the, “How do people currently solve a problem?” is critical for achieving Product/Market Fit. Otherwise, after it’s too late, you’ll find that your audience is already satisfied with a different way of solving the same problem and that your company, startup, or product has become redundant. To be more specific, here are some great times to ask the question (Figure 3.3):

• When you strategize: Exploring how people solve a problem today helps you come up with a great idea tomorrow, since the best predictor of future behavior is current behavior. Even if you have a product idea, figuring out the problem it solves might lead you to improve it significantly.
• When you execute: Keeping your eyes open even during the development of your product idea can help validate it, fine-tune it, or pivot to a better strategy, if needed. Or perhaps even invalidate it if you find the idea is no longer relevant.
• When you assess: Putting your product aside for a moment and bringing fresh eyes to the field to observe how people behave without your product can help prioritize features on your roadmap.

Answering the Question with Observation#section4

One of the most reliable ways to answer the question “How do people currently solve a problem?” is through observation. While not an easy technique to apply, observing people in their natural context of using products or services can take you a long way toward deeper learning about a real problem. Observation involves gathering data in the user’s environment, so it is the science of contextualization. Observation can be referred to by many names, including:

• Field observation
• Field study, fieldwork, field research
• Contextual inquiry
• Guided tour
• Fly-on-the-wall
• Shadowing
• Ethnography

The different names sometimes indicate how much interaction happens between the participant and the moderator. Fly-on-the-wall and shadowing hint at no interaction, while guided tour and contextual inquiry might indicate there’s more of a conversation going on. The only exception to the list of names is ethnography. In classic ethnography, the researcher (that’s you) immerses herself among the group she is studying and joins its activities. For example, if family cooking is of interest, the researcher joins a family and cooks with them rather than interviewing them, while simultaneously observing what family members do. The truth is that it doesn’t really matter what you call it. As long as you are observing a person in her natural environment, you are in the observation business.

There are five important pillars for observation:

1. Observing: Watching people as they go about their daily lives at home, work, in between, or wherever is relevant to what the product team is interested in. Observing will help you uncover not only what happened, but also why it happened.
2. Listening: Learning the language and jargon people use in their own environments, as well as witnessing conversations they have with others. Listening to people’s jargon has an extra benefit of identifying words they use to describe things. For example, when using online banking, many people struggle to find mortgage information because banks use the word loan to describe a mortgage. Uncovering user jargon in observation can help you identify language to be used in your product.
3. Noticing: Paying attention to a variety of behaviors and occurrences that might have significant implications on user needs. Just standing there watching what people do can be a challenging and overwhelming experience if you don’t know what to look for. Looking for and paying attention to behaviors such as routines, annoyances, interferences, habits, etc. turns “just being there” into an effective design tool.
4. Gathering: Collecting different things (aka, artifacts) that people use or create to complete certain tasks might signal user needs or missing features or products. For example, an artifact you might find useful if you were conducting an observation in a grocery store would be a person’s grocery list.
5. Interpreting: Figuring out what the observed behavior means and why the person is doing it that way.

Why Observation Works#section5

Observation is an effective user research technique that carries the following benefits:

• Identifying new features or products
• Validating/invalidating team assumptions about users
• Identifying problems people might have
• Understanding user goals
• Understanding people’s workflows

Other Questions Observation Helps Answer#section6

Other than the “How do people currently solve a problem?” question, observation is a great method for answering the following questions as well. If you ask yourself any one of these questions, observation can help you get an answer:

• Is there a need for the product?
• Why are people signing up and then not using the product?
• What are some feature ideas our customers have?
• How do people choose what to use among similar options?
• How do we make using this product a habit?
• Does the product solve a problem people care enough about?
• Which customer needs does the product satisfy?
• How do we define the right requirements for the product?
• How will the product solve people’s pain points?
• Which features are most important?
• Should we build [specific feature]?
• Who are the product users?
• What are the different lifestyles of potential product users?
• What motivates people?
• What jargon do people use to talk about a specific topic?

How to Answer the Question#section7

The following is a how-to guide that takes you step-by-step through the process of using observation to answer the question “How do people currently solve a problem?”

Step 1: Find eight research participants.#section8

Finding participants for observation raises a limitation of the method that you should be aware of. Naturally, when you want to observe people, you should be right next to them. This means that you and your participants should be in the same location. However, several situations might happen based on the location of your target audience:

• Your target audience resides in your location. No problem. Carry on.
• Your target audience resides in your location and in other locations. Try to make an effort to travel to other locations for observation. If traveling is not an option, observe people in your location and apply other research techniques with people in other locations (such as interviewing, experience sampling, or a diary study).
• Your target audience resides in other locations, some (or all) very far from where you are located. If your most important audience is far away from you, make an effort to travel for observation. If traveling is not an option, you can either be creative with remote observation (ask your participants to broadcast live from their phone as they go about their lives) or apply other research techniques (such as interviewing, experience sampling, or a diary study).

Recruiting participants is the greatest bottleneck of user research. Start as soon as you can. Chapter 9 guides you through how to find participants for research through social media. The following are the key steps in this process (previously shown in Chapter 1):

1. List your assumptions about participant criteria (e.g., business traveler).
2. Transform participant criteria into measurable benchmarks (e.g., travels for business at least three times a year).
3. Transform the benchmark into a screening question or questions (e.g., How often do you go on an airplane?). If a person chooses the “right” answer, he’s in. If not, he’s out.
4. Craft a screening questionnaire (also called a screener) that you can send people. (Here is a sample screener.)
5. Pilot-test the screener with a couple of people and make improvements.
6. Identify relevant social media groups, pages, communities, and hashtags where your audience is likely to linger and post calls to take your screener.

Observation generates huge amounts of rich data, somewhat similar to the amounts you might collect in interviewing (see Chapter 2) or diary studies (see Chapter 4). These large amounts of collected data directly affect your choice for the number of participants you observe in the study. As in other qualitative methods, keep this number low and digestible. Eight participants is a good number. More than that requires more time or hands when it comes to analyzing data and coming up with results.

Step 2: Prepare a field guide.#section9

Before you go to the field to observe people, prepare a field guide that will help you focus on your goals and support your data collection. The first thing on your field guide is a short list of research questions. Research questions are questions that will be answered during the study. Don’t confuse them with questions you might ask research participants. They are not the same. Research questions are questions you or the team has. They indicate a knowledge gap you want to fill in with insights. As a ballpark estimate, you should probably have around five research questions. Research questions will help you during observations and will guide you into what parts you need to pay most attention to. For example, here is a list of research questions you might have prior to observing someone during grocery shopping:

• How do people choose which items to buy?
• What are the items people have most difficulty in finding?
• What is the primary challenge people have when grocery shopping?
• In what situations do people use their smartphone to support their grocery shopping? What is the motivation behind it?

The primary goal of a field guide is to help you capture the necessary data during observation sessions. The level of detail in a field guide depends on your level of experience conducting observations and how much structure you need in taking notes. Here are two approaches you can pick from:

• Less structure: Create a list of things to pay attention to and look for while keeping an open mind about new, sometimes surprising, things that will reveal themselves to you during observation. If you are a person who trusts her intuition, do just that and allow yourself to add to the list or deviate from it as observation progresses. Here is a sample such list for a grocery shopping observation:
• Problems we want to observe: Challenges in grocery shopping such as deciding what to purchase, finding items, or wasted time.
• Problem could occur when: Participant stalls, doesn’t know where to go, asks for help, looks repeatedly at grocery list, or calls spouse.
• Details to be recorded: Full description of observable problem, time spent on solving problem, participant’s decision tree, participant’s motivation to solve the problem, external factors affecting problem or solution (technology, other people), or chosen solution.
• Back-up strategies if the problem/behavior is not happening: Ask for a retrospective demonstration as close to the real thing as possible, ask what was challenging, ask participant if what happened was typical; if not, probe to explore past challenges.
• More structure: Structure your note taking and prepare as many placeholders as possible to save time during observations and make sure you pay attention and document everything you need. Structure the field guide so that it begins with simple, specific questions your participant feels comfortable answering. Then go broad and list questions or behaviors to look for that are broader, and finally after setting the context for more targeted questions, finish with deeper probes that will help you with your innovation challenge. Figure 3.4 is a screenshot from a sample field guide for grocery shopping you can use as a reference .

Step 3: Brief observers.#section10

Assign people in your team who join observation sessions with roles such as note-taker, photographer, and videographer. Conduct a brief meeting during which you give observers a short background about the participant and about what’s going to happen. Dedicate enough time to discuss expected participant actions and your reactions. An observation might present uncomfortable situations for participants, moderators, and observers.

In a typical observation, a small team, consisting of a moderator and two to three observers, visits a participant in her natural environment such as home or work. The challenge of the team is to overcome situations that might cause participants or other stakeholders (e.g., a participant’s spouse or manager) not to cooperate or to put a complete halt to the session. The observer brief is a great opportunity to discuss potential participant reactions during observations and ways to overcome them with appropriate reactions by the team. The more prepared the team is for these situations, the higher the chances are to defuse them.

Following are several situations that might come up during observation sessions and how you should react. The purpose here is not to intimidate you, but to have you be aware of and prepared for unexpected situations that might happen.

1. The participant is reluctant to share what’s on his computer/phone. (“Let me go to my bedroom where the computer is, find the answer to your question, and quickly get back here.”)
   • Prevention: Ask more personal questions or requests when you are getting on well with the participant. Feel the energy. Spend time building rapport and try to make these requests later in the session.
   • Explain why you are so interested. Have a story prepared for how you are trying to help people, how important your research is, and how many people could benefit.
   • Don’t take photos of the participant’s screen.
   • In your notes, indicate “return to topic” next to what happened throughout the early parts of the session and try again later if you feel it is right.
2. The participant refuses to hold the session at the most relevant location. (“My office is really small so I don’t think we can all fit.”)
   • Ask if you can pass by his desk or study room or bedroom (wherever the relevant location is) on the way out just to get a feeling of his home or workplace (“That’s totally fine. We don’t mind sitting on the floor.” Or you could ask “Can we pass by on our way out just to get a feel?”).
   • If appropriate, ask if you can take a picture of the relevant room or area.
   • Don’t push too hard on this.
3. The participant thinks too many people showed up. (“Wow, a whole crew!”)
   • Apologize if it wasn’t communicated clearly enough in advance.
   • Make the participant feel like a superstar: “You have been specially selected because you are so interesting and important. The team wants to model its product on you because you represent the ideal customer so much, but, of course, it will only be worthwhile if you are really honest and act completely naturally.”
   • Explain that it is important for team members to attend since they want to learn from her knowledge and experience.
   • Explain that observers will not be very active during the session.
   • Worst comes to worst, ask one to two observers to leave. Best thing to do is to completely avoid this situation altogether. Have a maximum of three people show up for observation. If more people want to observe, create an observation shift schedule.
4. The participant refuses to be photographed or recorded. This refers to a situation when the participant initially agreed, but when you actually take the camera out or even after you take a few pictures, he is clearly uncomfortable with it or directly asks you to stop.
   • Prevention: Mute the camera’s “click” sound and use a smartphone, which is less intimidating than a huge SLR camera.
   • Prevention: Let the participant know you will ask permission every time you want to take a photo, and he can say no at any time. You will respect his privacy and understand that certain situations should not be recorded.
   • Explain that photos are extremely important for this study since they show important decision-makers in your company that there are real people behind your recommendations.
   • Offer to show him all the photos you took when the session is over and allow him to delete ones he is not comfortable with.
5. The participant does not answer the questions.
   • Prevention: Ask the question and stay silent. Your participant will feel an urge to fill in the vacuum with an answer.
   • Try to find out if there is anything that makes her uncomfortable. If you are in a team, you could take the participant aside for a one-to-one meeting. This could be to find out if she feels uncomfortable or to get more detailed data on your own.
   • Tell a story. It can either be about you (something relevant) or about another participant (anonymous, of course) who had similar opinions or experiences. It can show that there is nothing embarrassing about the situation, because you come across it all the time and you can relate to it personally. It also helps to make you more human, likeable, and friendly.
   • If the participant insists, move on.
6. The participant invites others to join.
   • Find out who the others are and why they want to join. It might become an advantage by having additional perspectives and seeing how the person behaves in front of friends.
   • Explain this is an individual session and that you cannot have others join it.
   • Explain it is important for you to get his personal perspective.
   • Give up if he insists. Sometimes, it’s even the right thing to do.
7. The participant says she doesn’t use X.
   • Explain that the products she is using are not the focus of the session.
   • Say, “Interesting. What do you use to solve for ____?”
8. The participant’s manager/spouse is not aware of what’s happening.
   • Repeat the introduction sections dealing with goals, procedure, confidentiality, and documentation to the manager or spouse.
   • Apologize that the other person wasn’t informed of this in advance.
   • Ask for permission to continue.
   • If you are kicked out, leave quietly. Make no fuss about it.
9. The participant is nervous, stressed, or physically uncomfortable.
   • Telling a story can help break the ice. Sometimes pointing out that he seems nervous can make it more awkward, but changing the energy with a story can be a stealthy way to deal with it. Talking to the participant like a friend will help make it seem less like research. The story could be about another participant and how she helped to develop a great solution that solved real-life problems others had.
   • Offer to take a short break.
   • Answer questions the participant has about the session.
   • Sneak in another mention of your confidentiality commitment.
   • If you feel he is at a distress level that is too much, stop the session, politely apologize, and leave.
10. The participant wants to finish earlier.
    • Try to figure out why and answer her questions.
    • If she mistakenly scheduled a conflicting event, ask how much time you have, promise to finish on time, and keep your promise.
    • Reschedule for another day. Split the session, but still get all of your data.
11. You want to finish a few minutes later.
    • As soon as you realize you need more time, ask the participant if that’s okay. Don’t do it when time is up.
    • Tell the participant how much more time you will need. Keep your promise. Don’t ask for more.
    • If the participant doesn’t agree, thank her and finish on time even if you didn’t get to ask everything you wanted to.

Step 4: Practice!#section11

It’s a short yet critical step. Gather the observer team, recruit a fake participant (a colleague perhaps), and practice observation for 15–20 minutes. This will help with setting expectations, getting used to paying attention to important things, note-taking, taking photos, recording video, invading personal space, not bumping into one another, and other small logistics stuff that would prevent you from wasting precious time or looking stupid and unprofessional.

Step 5: Gather equipment.#section12

Whatever equipment you decide to take with you, keep the following in mind:

• Small cameras: Large video cameras with tripods intimidate participants and make them change their behavior. It’s enough that three people are there to look at them and document their every move. Don’t add to that feeling. Consider using a GoPro (or a similar small size) camera. The size of the camera creates a better feeling than showing up with a four-foot tripod and a high-quality, full-size video camera. There’s no rule you have to video record observation sessions. Consider video recording if there are team members who cannot attend sessions yet are interested in watching them, or if you feel you will need to come back to your team with stronger evidence.
• Smartphones as cameras: Instead of large or dedicated video cameras, consider using your smartphone as a video and a still photo-recording device. Most people are more or less used to seeing a phone held in front of their face.
• Quiet cameras: Silence your camera. Clicks and beeps intimidate participants and remind them they are being recorded, which will cause them to deviate from their natural behavior.
• A shot list: This is a list of photos you hope to take—the person’s portrait, an artifact, a contextual shot of their space, etc.
• Extra batteries: Batteries drain. Take a lot of spare ones for everything that requires charging.
• Chargers: Take charging equipment in case you will have an opportunity to charge while in the field. Cables, plugs, dongles, and power splitters can become extremely handy. When you get to an observation location, immediately survey the area to identify power sockets. If there are any, ask for permission to use them. Do that after you establish rapport (see next step). If you don’t expect to have a power outlet in your observation area, consider taking with you a charging device or even a fully charged laptop from which you’ll charge your other devices.
• Memory cards: Make sure that you have spare memory cards for phones, cameras, and videos. Remember to check them and take previous data off so that you have maximum space. Try to calculate how much you will need—for example, if one hour of video recording takes up 4GB, then take enough memory cards or change the recording quality. Don’t run out halfway. If you are recording in HD, then consider the implications of how you will store and transfer files. How much space do you have on your laptop if you’re in the field and need to transfer every day?
• NDA: This is a non-disclosure agreement that describes the confidential aspects of your study as well as why and how you use the recordings and data you collect. There might not be a need to ask participants to sign an NDA if you are not showing them anything confidential. However, sometimes, you might want them to sign one to protect the method you are using and the questions you are asking, which might indicate a confidential aspect of your business.
• Incentive: If you promised the participant money or a gift, make sure you take these with you.
• Audio recorder and lavalier microphone: While a big bulky camera is intimidating, a recorder and lav is quickly forgotten and ignored by participants.
• Save juice: Make sure that you limit your device usage during observation days to observation needs. No Twitter, Facebook, or texting with friends. Watch those cat videos when you’re back at home.
• Office equipment: Take notebooks, pens, and pencils in case all hell breaks loose. I know it’s hard, but I guarantee you will quickly learn what to do with them in case all of your device batteries are drained.
• Post-it notes: Taking notes during observation sessions on Post-it notes can save you precious analysis time later on. Get a pack of colored Post-it notes, one color per participant. Make sure that you have enough of them so that every observer can use a pack in each observation session. So, for example, if you plan to observe 6 people and you have a small team of 3 observers, you will need a minimum of 3 blocks of red Post-it notes, 3 blue blocks, 3 yellow, 3 green, 3 purple, and 3 orange blocks.
• Mints and Water.

Step 6: Establish rapport.²#section13

When people agree to participate in “research,” they imagine meeting someone who spends all his time in a lab, wearing a white robe with a name tag, holding a writing pad, experimenting with rats all day long, and wearing rectangular glasses on half his nose.

Then you show up.

All the things you say in the first five minutes—every word, each of your voice intonations, the things you do, your gestures and body language—have a tremendous, sometimes underestimated, effect on your participant’s behavior throughout the entire interview session.

The following things to say and do will help you create rapport with your participant. Your participants will perceive you and this whole weird situation of interviewing in a more positive way. Not completely positive, but more positive. Most of these things are also true for first dates. There’s a good reason for it.

1. Smile. It’s our most powerful gesture as humans. Research shows that smiling reduces your stress levels, lifts your mood, and lifts the mood of others around you.³ It will also make you live a longer, happier life, but that’s for a different book.
2. Look ’em in the eye. When you look someone in the eye, it shows you are interested. When you do it all the time, it’s creepy. Try to look your participants in the eye during the first 10 minutes of the session for at least 30% to 60% of the time—more when you are listening and less when you are talking.
3. Avoid verbal vs. nonverbal contradictions. When your participants identify such contradictions, they will be five times more likely to believe the nonverbal signal than the verbal one.⁴ For example, if you say to participants you will not use the study’s video recording publicly while you wipe sweat from your forehead three times, they are going to think you are lying. When you are sending inconsistent messages, you are confusing participants and making them believe you are insincere.
4. Listen. From the moment you first meet your research participants, listen carefully to every word they say. Show them you care about what they have to say.
5. Say thank you. Keep in mind they volunteered to help you. Agreeing to have someone follow you, look at you, take notes about everything you say and do, sometimes in your own home is something you should appreciate and be grateful for. Don’t forget to thank your participants from the very first moment. They should know you really mean it.
6. Dress to match: If you normally wear a suit and you’re meeting a customer in her home who may be wearing sweats and a t-shirt, you might come off intimidating. Likewise, if you are a hoodie and sneakers type meeting someone in a professional setting, then dress to match that setting. You don’t want to be disrespectful. Always ask the participant if he would like you to take off your shoes in his home. You are his guest.
7. Check your appearance: Make sure that you don’t have something stuck in your teeth. That piece of gunk will make you look unprofessional, which will not help with establishing rapport, as stated in Chapter 2.

Step 7: Obtain consent.#section14

As stated in Chapter 2 and bears repeating, informed consent means that your research participants are aware of their rights during a research session. It is not about whether they sign a form or not. It’s about having people truly understand and agree to the following:

1. They agree to participate in the research session.
2. They understand what is going to happen during the research session.
3. They understand how data collected during the research session will be used.
4. They understand they are being recorded.
5. They agree to being recorded.
6. They understand how the recording will be used.
7. They understand that their identity and privacy will be kept.
8. They understand that participation is voluntary.
9. They understand they have the right to stop the session at any point.
10. They agree to raise any concerns they might have immediately.
11. They have an opportunity to ask questions before the session starts.

Why You Must Obtain Consent#section15

As I said in the last chapter, I can give you my spiel about how applying the Scientific Method⁵ is important and that obtaining consent from research participants is a key part of it. But I’m not going to do that. Instead, I’ll just say that obtaining consent is the right, ethical thing to do even if you are “just talking with people.” Half-assing your research ethics, means you’re half-assing your learning process, means you are half-assing your product development. Although informed consent sounds like a term taken from a court of law, it is not. It is the fair thing to do and the best way to treat people who happen to be your research participants.

Step 8: Collect data and pay attention.#section16

The hardest thing to do during field observation is to pay attention to everything that is going on in front of your eyes. You might not realize it, but observing how humans behave generates tons of rich data. It is sometimes challenging to notice when something important happens. Your guide as to what to look for is to stay focused on the reason that you are running this research in the first place and the goals you have set for user research. Focus on things related to your goals. When you are observing a study participant, look for the following occurrences:

• Routines: Routines are things that seem to be regular actions the participant is following. For example, each time a new work-related task comes up, the participant logs it on a spreadsheet that he has created. This routine can later turn into a feature in your product.
• Interactions: Follow her interactions when a study participant uses a certain product, tool, or service, or when she converses with another person. For example, when a study participant doesn’t understand a certain word, she might use an online dictionary to figure it out.
• Interruptions: An interruption might occur when a study participant stops a task or breaks its continuity either because he has decided to do so or because another person caused it. For example, when a phone call comes in and diverts the study participant from what he is doing. Note that it is intuitive for the researcher to ignore these interruptions, yet in many cases they can teach you a lot. Life is not always “clean” of interruptions so we must understand them.
• Shortcuts/workarounds: When a study participant chooses a shorter alternative, it is sometimes an indication of a small problem to pay attention to. For example, when instead of writing something down, a participant takes a pen and marks an X on the back of her hand. What that means for your product or people’s needs is not clear when you observe it. Yet this behavior might relate to a different one you observe that might make sense later on.
• Contexts: Context occurs when a certain action or behavior is demonstrated in a different manner because of the environment in which it happens. For example, when a participant does not take a note on his smartphone because of direct sunlight that makes it hard for him to see anything he types.
• Habits: These are behaviors participants demonstrate that are almost automatic. For example, scribbling something with a pen to make sure it works, even though it is brand new.
• Rituals: A ritual is an established sequence of behaviors in a particular context that cannot be explained purely in terms of being functionally necessary. It’s almost optional or voluntary; for example, buying a drink if it’s someone’s birthday and singing happy birthday to them.
• Jargon: Paying attention to the language and jargon people use in their own environments, as well as witnessing conversations they have with others, is extremely helpful in empathizing with them and uncovering their needs. Using the unique language people use when they talk about different things will prevent you from using language your audience doesn’t understand in your product or service. For example, if a person you observe keeps referring to a mortgage, that’s a good signal for you to use this label in your online banking app rather than calling it a loan. You might learn that people interpret the term loan very differently than how you or your team does. It’s also a good cue to mimic their language in the observation session in order to appear less different and to build rapport.
• Annoyances: Annoyances are obstacles that keep people from completing their tasks or achieving their goals. An annoyance would not necessarily prevent them from reaching their goals, but it would make them angry, frustrated, overwhelmed, or disappointed along the way. For example, a person might get annoyed while filling out an online form while dealing with noise from a nearby room.
• Delights: The things people enjoy can teach you a lot about what they need. Many people perceive research as an activity that uncovers problems and frustrations. That’s partially true. Uncovering things that delight and work well for users can go a long way toward developing great products. For example, you might notice people who are satisfied by in-field form validation instead of validation done after submitting the form.
• Transitions: When people move from place to place, it’s a great time for them to share things that might become invaluable—especially when they think the research session is over or on a temporary pause. For example, if you observe someone taking notes in a certain classroom, pay extra attention to what happens when the class is over and until you part ways with the participant.
• Artifacts: Artifacts are tools, services, products, any other thing that people use to complete tasks, or seemingly useless yet meaningful objects (such as rubber duckies for developers). Your job is to pay attention to the usage of artifacts, and if possible, collect or document them. For example, if a person is taking notes while using a LiveScribe pen and notebook, that’s an important artifact to take note of, no pun intended.

Here are some additional pointers to note in order to get the most out of observation:

• Approach each observation session with an open mind. You’ll find that in many cases, you invalidate your initial assumptions about people and their problems and reach insights you never realized.
• Have a conversation with the person, not an interview. Don’t just go through the motions of what you planned. If you feel there’s something to talk about that’s worth the time, make the time for it. Don’t feel you must stick to the script.
• Let your participants be. Don’t interrupt or talk over them. If you do, they’ll avoid sharing additional things with you, and you might be missing key insights.
• Pitch your level of knowledge to match the participants. Try not to make them feel like you’re more knowledgeable than they are.

As you observe, that’s a lot to track and digest. You need a lot of practice to get it right. Don’t worry, though. Even if you miss a few things, you’ll still get to learn many valuable lessons and you’ll get better in time.

Step 9: Debrief.#section17

A common mistake is to assume that every observer interpreted the same things you did or placed similar value on certain observations. Debriefs and syntheses are a process of creating a shared understanding so that the team can move forward in a unified direction. Debriefs will help you capture your insights while they are still fresh in your mind and will decrease the load of analysis and synthesis that awaits you after all of the observations are completed. There are two types of debriefs, the quick debrief and the daily debrief.

Quick Debrief#section18

Shortly after you are done with each field observation session, conduct a quick debrief with observers. Do it in the lobby of the building, in the train, cab, or bus, in a park, on a bench, wherever. The most important thing is to conduct the debriefing shortly after the session ends so that things are still fresh in your mind. This will also prevent you from getting confused if you run several sessions in one day. In addition, take five quiet minutes to yourself and write a short paragraph that summarizes the session.

During the quick debrief, ask yourself and the observers the following four questions (inspired by IDEO’s human centered design kit⁶):

1. What did the participant say or do that surprised you? Were there any memorable quotes?
2. What mattered most to the participant?
3. What stood out during this session? What are the big lessons?
4. What should we do differently in future sessions?

If you’d like to try this debriefing technique, run a quick debrief for the second field observation video you watched in the sidebar after Step 8.

Daily Debrief#section19

When you conduct several observations per day for several days (e.g., two half-day observations every day for three consecutive days), gather the team at the end of each day in front of a large wall and run the following exercise (aka, affinity diagramming):

1. Put all of your Post-it observations on the wall.
2. Organize them into temporary, logical groups. The groups can change from daily brief to daily brief.
3. If you used unique Post-it colors per participant, you’ll notice very quickly which groups of observations were popular among different participants and which ones were only observed with one or two participants (see Figure 3.5).
4. Take photos of the wall (these Post-it notes tend to fly off).
5. Log groups and items into a spreadsheet.
6. Continue working on affinity diagramming until data collection is completed.

What to Do with Photos#section20

Photos you took during observation sessions provide inspiration, visual context, and sometimes, supporting evidence for your findings. The idea is to fill your design space with inspiration from the field. Here are some ideas:

• Organize photos based on groups of observations you identified during debriefs.
• Curate topical photo galleries.
• Print a photo of each participant to remember that person.
• Cover a wall or board in photos from the field and tag them with observations.

Step 10: Analyze and synthesize.#section21

There is no one way to make sense out of observation data you collect. That said, affinity diagramming combined with storytelling is a straightforward approach that seems to work for teams. Here are the steps to complete an affinity diagramming and storytelling exercise:

1. Complete the affinity diagramming exercise you started in the daily debriefs. Sort all of the observations into groups and give each group a name. Alternatively, you can do that by implementing the KJ Technique.⁷
2. As a team, select the most important and meaningful groups.
3. Per group of observations, write a short story that describes a future scenario of a person using a product or feature that doesn’t exist yet. The story can be very short—about 150–200 words. Base the story on a problem or need you identified during observation.
4. Share the stories with the team, gather feedback, and get agreement and shared understanding.

Other Methods to Answer the Question#section22

While observation is a great, immersive way for answering the “How do people currently solve a problem?” question, the following are two additional methods for answering it. Ideally, if time is on your side, a combination of two to three methods is the best way for uncovering insights to help you answer this question.

• Interviewing is a research activity in which you gather information through direct dialogue. It is a great way to uncover and understand people’s feelings, desires, struggles, delights, attitudes, and opinions. Interviewing people whom you know to be your target audience (and those you think are not) is a great way to get to know your users, segment them, design for them, solve their problems, and provide value. An interview can be held in person or remotely over a phone or some kind of a video conference. Chapter 2 guides you through conducting interviews for uncovering needs.
• In a diary study, participants document their activities, thoughts, and opinions and share them with you over a period of time. A diary might be a record of their experience using a product or a means to gain understanding of ordinary life situations in which products might be usefully applied. Diary studies are best for learning about more complex processes. Chapter 4 walks you through conducting a useful diary study.

Note On Resources#section23

Access the online resource page for observation on the book’s companion website at leanresearch.co. You’ll find templates, checklists, videos, slide decks, articles, and book recommendations.

Observation Checklist#section24

• Find eight research participants.
• Prepare a field guide.
• Brief observers.
• Practice!
• Gather equipment.
• Establish rapport.
• Obtain consent.
• Collect data and pay attention.
• Debrief.
• Analyze and synthesize.