UXBoost applies mixed methods UX research to understand research problems and propose solutions comprehensively. We've found this approach incredibly useful and want to share a checklist to help you plan your mixed methods UX research. Before diving in, let's revisit what mixed methods UX research is to decide if it is suitable for you.
“Quantitative gave us a large general surface picture, qualitative gave us the in-depth pictures. You've got to say both pictures are probably valuable.” - John W. Creswell
According to John W. Creswell, the founder of mixed methods research, mixed methods research is an emerging approach combining statistics (Quantitative) and stories (Qualitative) to comprehensively understand a research problem. The core assumption is that combining both methods will provide a better understanding of the problem than in isolation of each other.
Mixed methods UX research is not simply the collection and reporting of quantitative and qualitative data separately, as this fails to combine and synthesize the two types of data and integrate them in mixed methods research.
Mixed methods UX research is rigorous. It would help if you considered how you want to design the study and how to collect and analyze data rigorously. There are 3 basic mixed methods designs that we discussed in the previous case study, namely Explorative Sequential Design, Explanatory Sequential Design, and Convergent Design.
Goals are the heart of the whole research project. Setting clear goals with stakeholders, whether internal or external, is critical. The goals will be the key to determining the project's research methodology.
You might wonder when a mixed methods is well suited for your UX research. Here are some questions you can ask yourself:
Mixed methods UX research is well suited when you need a comprehensive understanding of a research problem. It might also take more time to conduct than quantitative or qualitative research alone.
✨Pro tip:
Once you have the research goals, you can determine what type of research design suits you. The following are the 3 main types of mixed methods research designs:
Exploratory Sequential Design focuses first on qualitative research. To gain a better understanding across a larger audience, you would use qualitative insights to build the quantitative study instrument. This approach works best when you are working on research questions with many unknowns and need some direction to frame the quantitative research questions or even the choices.
Explanatory Sequential Design is the opposite. You would focus on quantitative insights such as existing user base data, analytics, or survey data before diving into the qualitative research as a follow-up. You will interpret quantitative results using qualitative data.
The Convergent Design would be quantitative and qualitative research conducted simultaneously but separately. This approach is used to merge or compare two datasets for any discrepancies. The researcher would analyze the data to determine how these two datasets support or diverge.
✨Pro tip: You can read our case study to understand how to apply Exploratory Sequential Design Research.
If you focus on the quantitative analysis of user base data, analytics data, research will take place in your own setting. On the other hand, survey or quantitative usability testing happens at the participant’s own location.
For qualitative research, consider the format that fits your situation. If you plan to conduct a user interview or usability testing, you must consider whether you want to conduct the study in person or remotely.
The primary purpose of this Mixed methods UX research is to determine whether the results are comparable or diverge. Statistical tests will be in place to test the validity of the data. Please keep in mind that things can get complicated quickly due to the validity of the data.
✨Pro tips: Be clear on your research intent and how you would systematically integrate and analyze different datasets.
Mixed methods UX research can take longer. Planning the whole research timeline carefully helps keep the study on track. According to our experience, it takes 3 to 4 weeks to complete the study, including participant recruitment. Nevertheless, it may vary depending on the scope or complexity of your research project.
Once you have a clearer picture of what and how you want to do the research, document it in a test plan. This will serve as a guide to communicate how to carry out the research and as documentation for future research. A test plan should consist of these key elements:
✨Pro tips: Ensure your test plan is clear, concise, and easy to understand.
For qualitative research, it is recommended to have 5 participants per target group for the best return of investment. As for quantitative research, the Nielsen Norman Group recommends having at least 20 to 30 participants in each target group. However, quantitative research, like surveys, might need a larger sample size to achieve statistical significance.
You must also pay close attention to the participant criteria and ensure consistency throughout the study. Keep your screener survey brief. Always avoid “yes” and “no” questions and leading questions that will hint to the participants what the “correct answer” might be.
✨Pro tip: Start participant recruitment when you are ready with the research plan and screener survey. Recruitment can take time, primarily if you are recruiting very niche participants.
Never skip pilot testing in any research project unless you want to experience feeling-at-war troubleshooting issues during execution. Pilot testing is essential for us to fine-tune the research instruments and anticipate the challenges and time requirements of the participants.
✨Pro tip: Recruit a pilot tester who meets the study's participant criteria whenever possible. If a pilot tester is unavailable, consider seeking assistance from a friend or colleague not involved in the study. Some pilot testing is better than none at all!
Once you complete the research, it is time to converge! It takes a coordinated effort to achieve design improvement. Invite your team members or even stakeholders to be part of the research. Allowing them to observe the session can reduce debate time and create more buy-in.
✨Pro tip: Brief your observers clearly before the session. You want to avoid any unnecessary disturbance or surprise that will impact the study.
I hope you now better understand how to conduct mixed methods UX research. We’re giving away a free Mixed Methods UX Research Checklist to help you plan your research better!
Many thanks to Celine Lee, who helped me create the beautiful checklist.
Read More:
.png)
Valentinie is a UX researcher with a background in Cognitive Science. She has experience researching different industries, from E-government services, credit bureau, UX learning platforms, blockchain, food delivery, etc. For her, there is nothing more rewarding than seeing how user research can inspire and shape products that solve real user problems.
