Today’s post will be the first in an ongoing series about biases and other traps that can affect the results of your market research. These can come into play at any point in the process, from writing your survey or interview guide, selecting participants, or when conducting your analysis. Being aware of them is necessary not only for those conducting the research, but also client-side employees who are tasked with evaluating the work of a research supplier.
We’ll begin by discussing a common bias in survey writing. Biases in survey work can be broken into two categories.
- Selection bias: Inaccuracies are due to the pool of respondents selected to take your survey.
- Response bias: Inaccuracies are due to how the survey is constructed (e.g. leading questions, skewed response scales).
Writing loaded/biased survey questions falls into the category of response bias since they can skew the results of even highly representative sample groups. It is easy to fall into the trap of wording questions that can elicit a positive or negative response or make a respondent feel there is only one “correct” answer.
- Example: Instead of, “How delicious was your meal on a scale of 1-5?” ask, “How would you rate your meal on a scale of 1-5, with 1 being very unsatisfied and 5 being very satisfied?”
Ethical researchers do not write leading/biased questions in an intentional manner. However, one’s own beliefs and experiences can unconsciously lead to wording a question with the assumption that respondents will answer in a way that matches the writer’s own opinion. Potentially skewing the results in one direction takes away the chance to obtain unexpected but helpful insights.
When designing a survey or interview guide, not only should you write with an open mind and an awareness of your own opinions, but also have others review your draft. Multiple eyes on a survey draft are essential for ensuring that an unconscious bias did not lead to any loaded questions. Everyone makes mistakes, so getting feedback from co-workers and your client will guarantee that the results of the survey are accurate and actionable.
At Vennli, our research team works closely with each other as well as our clients to make sure that the results of our studies can be acted upon. Reach out to learn more about how we can help you stand out from the competition with fast and agile market research.
