Want Better Answers? Ask Better Questions

When starting on the research phase for a digital experience, gathering valuable insights from users is crucial mandatory. By conducting effective user interviews, you can uncover the information needed to create successful, user-centered products. This article will walk you through the process of scoping your research, building rapport with your users, and crafting insightful questions that focus on past experiences, pain points, and behaviors.

Warning!

Before venturing any further reader, heed my warning: when conducting UX research, it’s crucial to approach the process with an open mind and a genuine desire to understand user needs and problems. Research should not be used merely to validate pre-existing assumptions or design decisions. It is there to dig deep into what is the pain points for users and help drive useful iteration. 

Scoping User Research Effectively

Before diving into user interviews and surveys, it’s crucial to properly scope your research efforts. In my experience, scoping user research can be one of the most challenging aspects of the process. You need a detailed understanding of the project’s goals, users, and the tasks before you begin – information you may not have until after the project kicks off. 

However, I’ve found some strategies that can significantly improve the accuracy of research scoping:

Get the details first

Instead of scoping based on a high-level overview, I recommend gathering stakeholders for a workshop to clarify business goals, user groups, key tasks, and potential challenges. This upfront work will inform your research plan and methods, saving you time and headaches down the line. 

This will also show people’s interpretations of the project and outcomes and serves as a great way to make sure people are aligned enough to help provide a coherent outcome. 

Involve UX researchers 

Always include UX researchers in the scoping process. They can ask the right questions, select appropriate methods, and provide realistic time estimates based on their expertise. Leveraging their skills at this stage is a game-changer. 

Note: This guide is meant to help any digital experience and this almost certainly applies to groups that don’t have UX researchers. That’s okay, we work with what we got. The UX research responsibility should go to whomever has the best listening and note taking skills. Not only does this compartmentalise roles BUT it also means you have an individual who defends a certain view point that may or may not align with the other design decisions.

Determine the right number of participants

Consider how many user groups, tasks, and repetitions you need to observe, as well as session length and recruiting difficulty. The total number of participants and session length will determine overall research time. It’s a balancing act, but getting it right is key.

For problem-discovery studies, which are more subjective, typically 3 to 20 participants are needed, with 5 to 10 being a good baseline. The number should increase with study complexity and product criticality, but decrease with design novelty. For comparative studies like A/B tests, which rely more on objective metrics, 8 to 25 participants usually provide valid results, with 10 to 12 being a good baseline. Larger group sizes are needed for statistically significant results. In general, there is no one-size-fits-all answer, but understanding the assumptions, limitations and risks associated with the participant group size is important when making the decision for a particular usability study.

Plan for analysis and deliverables

I can’t stress this enough – budget ample time for analysing research data. A good rule of thumb is 3 hours of analysis per 1 hour of research. Personally, I like to revisit data, turn that 3 hours into 2 or 3 “chunks” of time to a) make sure I didn’t miss anything and b) give me time to reconsider a different approach to data or see different connections. Also include time for creating deliverables to effectively share insights with stakeholders.

While it’s ideal to estimate the full scope of research needed, some clients may have a fixed budget or timeline. In these cases, I recommend scaling research to fit within constraints while still aiming to gather quality insights. Focus on the most critical questions and consider remote sessions to save time and cost. 

Remember, some research is almost always better than no research. By scoping effectively, you can ensure your research efforts provide actionable insights that move the product and business forward. In my opinion, mastering the art of research scoping is what separates good researchers from great ones. Invest the time upfront, and it will pay dividends in the insights you uncover.

Building Rapport

Asking the right questions is critical, but it’s only half the battle. To truly unlock deep user insights, you must also create an environment where participants feel comfortable opening up. This is where rapport comes in. 

Rapport is the positive connection between researcher and participant, built on mutual understanding, attention and respect. identified five key components of rapport: paying attention, personalising the interaction, being approachable, establishing mutual connection, and behaving professionally. By intentionally incorporating these elements, we create an atmosphere of trust that encourages users to share their experiences more fully.

For example, small actions like greeting the participant warmly, explaining the process clearly, and emphasising there are no wrong answers help participants feel at ease from the start. Active listening, eye contact and verbal/nonverbal cues of interest show the user you value their input.

When participants feel this sense of rapport, studies show they engage more fully and disclose more on sensitive topics. This can be invaluable when exploring user pain points, challenges, and emotional responses to your product. Strong rapport empowers users to open up about frustrations and share richer stories from their experiences.

However, it’s also important to be aware of potential pitfalls. The drive to maintain rapport can sometimes lead to socially desirable rather than fully truthful answers. Participants may be reluctant to criticise the product or share negative experiences if they think it will hurt their positive connection with you. One way I mitigate this is by emphasising that I didn’t design the user interface they may be looking at (even if I did). 

How to ask better questions

Now that the foundation for the user research scope has been set, the questions themselves can be formed and finalised. The key to insightful user interviews is crafting questions that uncover specific problems users have faced, their underlying motivations and emotions, and the larger context. Avoid leading questions or discussing hypothetical features. Instead, focus on the user’s concrete past experiences.

For example:

Instead of: “What do you think about the onboarding process we’ve designed?” 

Ask: “Think back to the last time you signed up for a similar product. What was the experience like? Were there any moments of confusion or frustration?”

The first question leads the user by asking for feedback on your specific design. The second question anchors the discussion in the user’s real experience, which may uncover pain points or expectations you hadn’t considered.

To help users articulate their process and avoid freezing up when asked a big question, break it down into a series of smaller, more specific questions. For example:

“Walk me through how you typically handle [task or situation].”

“What’s the first thing you do when you start working on this?”

“What happens next?”

“Are there any tools or resources you use during this process?”

“What’s the hardest part of this process for you? Why?”

“How do you know when you’re done or have been successful?”

This line of questioning helps the user break down their workflow step-by-step, making it easier to identify pain points, workarounds, and opportunities for improvement. It also allows you to gather concrete examples and stories rather than generalities.

Remember, the goal is to let the user share their experience and perspective, not to validate your assumptions. Ask open-ended questions, follow up with “why” to unpack their reasoning, and constantly look for opportunities to dig deeper.

User interviews are useful at various stages, including the idea stage, prototype stage, and when iterating towards product-market fit. Look for users who frequently encounter the problem you’re trying to solve, would benefit significantly from a solution, and have the budget to pay for it. To measure product-market fit over time, track how disappointed users would be if they could no longer use your product.

Conducting Surveys Effectively

Asking the right questions also extends to surveys. They are a tool to be used in addition to user research, potentially gathering effective qualitative data. They are never a replacement for user interview data. To execute them effectively they must have… 

Have a clear purpose 

Before launching a survey, critically assess if it’s truly the best method. Surveys are best for answering “how many” quantitative questions rather than “why” questions that require deeper qualitative insights. If you need attitudinal rather than behavioural data, a survey could work, but be aware of potential inaccuracies in self-reporting.

Keep it concise

Ruthlessly cut questions that aren’t essential. Aim for 10 questions max that can be completed in under 10 minutes. Less is better. Filler questions and long grid-style questions frustrate respondents and lead to unreliable data. Focus is key.

Avoid leading questions and priming

Pay close attention to question phrasing and order. Leading questions can bias responses. The order in which questions are presented can also prime respondents and influence subsequent answers.

For example:

Priming questions:

Q1: How important is it to have a balanced diet?

Q2: How many pieces of fruit did you eat yesterday?

Better questions:

Q1: How many pieces of fruit did you eat yesterday? 

Q2: How important is it to have a balanced diet?

Conduct cognitive interviews

Before launching, have a few people try answering the survey while thinking out loud. This will uncover points of confusion, awkward phrasing, or problematic questions you can fix. Crafting survey phrasing can be a tricky thing so don’t skip this step if you have the time. I can’t tell you how many times I’ve crafted what sounded like a great question only to watch a colleague perform mental gymnastics trying to read my 1st draft of survey questions. 

My conclusion 

By following these guidelines and recognizing surveys’ limitations, you can gather more accurate and actionable data to inform important product and business decisions. Effective surveys, combined with other research methods, help you keep a pulse on your users so you can build something they truly need and want.

Investing time upfront to understand the research context, select the right methods, and involve the right people will pay off in the long run. Proper scoping sets you up to conduct research that truly meets user and business needs. Remember, assumptions are the enemy of innovation. Only by genuinely listening to users and acting on their feedback can we hope to build products that solve real problems and stand the test of time.

Big thanks for the Cover Photo Credit: Alex Green. And Photos by Celpax and Christina @ wocintechchat.com on Unsplash