Strong Analysis Starts with Intentional Questionnaire Design
By Kana Feng
As quantitative researchers, people may assume that our skills are best reflected in our ability to review data, run statistical analyses, and shape those numbers into cohesive storytelling.
Of course, there is a lot of truth to this assumption, and this is undoubtedly an important part of what we do every day. However, the success of our analysis and reporting often begins with the groundwork we lay weeks, sometimes months, before we see a single percentage—during questionnaire development.
When clients come to us with a list of research objectives, our first step is to understand the context of the research and how (and by whom) the findings will be used. “What do we want to know about the target audience?” is a good starting point for drafting survey questions, but the decisions the research will support, the claims we want to make, and the subgroups we may want to analyze all inform questionnaire design.
A well-designed questionnaire:
1. Is easy for respondents to understand and answer, with little room for ambiguity in how they interpret the question text
2. Results in a clean dataset that requires little additional variable creation or other preparation before analysis
3. Provides the actionable data needed to answer the research objectives
For example, it’s common to need insights on your own brand versus competitor brands. Even a seemingly straightforward objective like this can lead to a number of important design decisions:
Considering all these factors helps determine how clear and organized the questionnaire will be and ultimately the actionability of the final results.
Another helpful exercise is to consider what we want the final deliverable to look like and say. For example, if we are capturing brand perceptions, do we need individual scores or ratings for each attribute? Or do we ultimately want to report on the “most important” attributes? Identifying what the end goal is reveals whether asking each attribute to be rated on a 5-point scale is best or if other methods like a ranking question or MaxDiff exercise would be a better fit.
A well-thought-out questionnaire is more than a data collection tool. It provides an essential foundation for the analysis, reporting, and overall success of the project.
Interested in learning more about W5’s POV on quantitative reach? Feel free to check these trending posts:
· Shorter, Sharper, Smarter: Why Short Surveys Outperform Long Ones
· Lessons from a Data Storyteller
· Modernized Best Practices for Survey Research
· “Please Tell Us Why…” | The Value of Open-Ended Survey Questions
· What is Perceptual Mapping, Anyways?