Researchers rely on a combination of both qualitative and quantitative UX research methods in product development. These two types of research are like two sides of the same coin. They are most powerful when used in combination to create products that are truly responsive to users’ needs and desires.
However, that doesn’t mean that the two methodologies are given the same weight. As we’ve previously discussed, qualitative research accounts for the lion’s share of UX testing. But quantitative methods play an important supporting role and should definitely be a part of your research toolkit.
In this article, we’ll help you understand the benefits of quantitative research and how they should fit into your overall approach to product development.
UX Research Methods: Quantitative vs. Qualitative
Quantitative research is focused on numbers. Its goal is to quantify a user’s experience in some way, typically by measuring a single metric. Think big surveys and analytics. Quantitative research yields hard data that is easily understood at a glance.
Qualitative research, on the other hand, is concerned with understanding why: why users behave a certain way, why they have certain motivations, and why they want or need your product to work a certain way. Qualitative research takes the form of observation and loosely structured interviews and results in nuanced interpretive findings.
Quantitative data is best at confirming that you have a specific problem. But only qualitative research can fill in the gaps and tell you why you have a problem and what you need to do about it. Put another way, quantitative data can provide an accurate and predictive picture of the usability landscape, but it can’t uncover the unarticulated user needs that underpin those realities.
Another difference to note: while qualitative research is best suited to very small groups of participants (5-10 is the sweet spot), quantitative research requires a much larger pool of participants in order to glean statistically significant data. In addition, quantitative researchers need proper training to appropriately synthesize the raw data that comes out of quantitative studies.
Because a large pool of participants is required, product developers sometimes dismiss quantitative research because they think it will be too costly or difficult to orchestrate.
But surveys, analytics, and customer intercepts are all inexpensive methods of collecting quantitative data that can be easily implemented. And quantitative research is only getting more cost-effective as technologies, such as those that allow remote testing and automated data collection, drive costs down.
When to Use Quantitative Research Methods
Qualitative research may be the bread and butter of UX testing, but quantitative UX research methods have an important role to play in the iterative product design process.
You should consider using quantitative research when you want to:
- Identify the existence of problems in a product. Just keep in mind that further qualitative research will be needed to fully understand the problem and arrive at a fix.
- Verify or ask a specific question about qualitative findings. This is especially helpful when your qualitative research points to the need to make a bigger business decision. But you can also use quantitative research simply to confirm whether your qualitative findings have pointed you in the right direction.
- Get buy-in from stakeholders around the need to solve a problem with further qualitative research. Remember: stakeholders are most likely to respond to hard numbers. Saying “1000 people reported that they had trouble registering for a class” is more compelling than simply saying “we have a problem with our class registration workflow.”
- Justify expenditures and demonstrate the ROI of previous qualitative research. In this case, you’re using quantitative data to justify and clearly define the benefits of your qualitative work. For example, you might use a survey to measure how much your product has improved after a round of updates based on qualitative research findings. In this case, quantitative research enables you to say something like, “after making X change, 1000 users reported that it was easier than before to register for a class.”
Quantitative Research Tools
UX researchers utilize a number of standardized tools to perform quantitative research. Here are a few of the most common methods and how they can help inform your product’s development.
- System usability scale (SUS score). This test is used to rate the efficiency and dependability of a product. Participants use a product and then answer a series of 10 standardized questions using a five-point scale from “strongly agree” to “strongly disagree.” Researchers translate responses into a score ranging from zero to 100. Scores above 68 are considered above average, while scores above 80 are in the top 10%. You might repeat the SUS score after making subsequent rounds of design improvements to the same product to gauge the success of those updates. If your SUS score goes up, that’s money well spent.
- User Experience Questionnaire (UEQ). Like the SUS score, the UEQ uses a standardized set of questions to evaluate users’ experience. The UEQ is newer and less widely known than SUS, but it has the advantage of measuring more variables than SUS. While SUS measures a product’s efficiency and dependability, the UEQ measure’s a product’s attractiveness, clarity, efficiency, dependability, and excitement.
- Net Promoter Score (NPS). Researchers use this single-question survey to understand the extent to which their customers are likely to promote their product. While the NPS offers an at-a-glance window into how favorably your product is viewed, it’s limited in usefulness because it doesn’t tell you why respondents feel the way they do.
- Affect Grid. An affect grid looks at how your product impacts users’ moods. To prepare an affect grid, researchers choose two pairs of opposing feelings (like depressed to excited and stressed to relaxed). Users interact with a product and then rate their feelings on a scale of one to nine. These responses are then mapped on a grid, with the hope that respondents will be grouped in the top right of the grid (excited and relaxed) versus the other quadrants (such as depressed and excited, which would imply an anxious feeling).
Despite their limitations, quantitative UX research methods have a lot to offer in the product development process. Understanding the particular strengths of quantitative data — and when to use it — is key to wielding this supporting player in the research toolkit efficiently and effectively.
