9 Best Practices in Survey Screening

wooden sieves

Most research begins by asking a fairly simple question: Who do we want to reach? Whether these people are consumers of energy drinks, owners of a hybrid vehicle, or those who binge watched Stranger Things (I qualify), they need to be accurately profiled and qualified for a study. What I’m talking about here is a staple of market research: screening questions.

In short, screening questions are placed at the beginning of a survey to determine who is – and isn’t – the right fit for a particular study. Those who “pass” the screener take the actual survey. Those who don’t, well, they are thanked and go on to help us out another time.

Here are some practical tips and best practices for creating screening questions.

1. Keep your screener short!

Ideally, try to keep the screening section to 10 questions or less – and this includes demographics. And think about the time the screener takes to complete. If it takes more than 2 minutes, think about how you can do better. We all need to be conscious of the time it takes someone to qualify for a study – with limited/consolation incentives. When someone is disqualified at the end of a long screener they become frustrated – and rightfully so. Be respectful of the people who help you inform important decisions.

2. Place necessary demographic questions at the front of consumer surveys.

Demographics that you need to qualify or set quotas for a study must be included up front. However, I advise not terminating a participant after a potentially sensitive demographic question. It’s best to let them complete the (short) screener section, or at least part of it beyond the demographics, before disqualifying them for the study. Non-essential demographic questions should be placed at the end of a survey, where only those who complete the survey see them. Do not take someone’s time for questions you don’t need if they don’t qualify. For B2B surveys, place essential firmographic questions such as industry and company size at the front and nonessential profiling questions in the main survey or at the end.

3. Start broad and narrow down to your audience.

Make no assumptions about the person starting your survey. Think of screening questions as a funnel that narrows down to your intended target audience.

Survey Design-01

4. Ask unaided awareness questions prior to awareness-of-brand lists.

Be careful about not priming or biasing participants with information you have given them.

Survey Design-06

5. Ask awareness questions prior to purchase or intent-to-purchase questions.

The key here is thinking about starting general and getting specific, just like you are trying to do as you narrow your audience. In the example below, knowledge is a prerequisite for brand consideration. If you flip the questions, your brand knowledge question answers will mirror closely your brand consideration questions – and this will be inaccurate.Survey Design-05

6. Eliminate leading questions.

Ask a question that doesn’t encourage people to answer a certain way. Don’t bias your results from the beginning.

Survey Design-02

7. Present questions with multiple answer options, as opposed to binary (yes/no) questions.

Binary screener questions tend to have a higher level of suggestibility – and tend to be leading (see above). Answers chosen from a list are more reliable, plain and simple.

Survey Design-03

8. Use simple sentence structure and avoid idioms, jargon, and acronyms.

Talk to participants as if they are humans. (Oh, that’s right, they are humans!) And I use the word talk intentionally. Write questions with colloquial language. Be simple and direct. Eschew phrases and terms that are not widely understood such as industry specific jargon and acronyms. Be clear. And probably don’t use the word eschew.

Meme for survey screening

9. Unless the list of answer choices is all-encompassing, include “Other” and/or “none of the above” options.

“Other” and “none of the above” options ensure participants are not forced into an inaccurate response by choosing a response that does not apply. I’m not a fan of using the ubiquitous “Other (specify)” unless you will actually look at this open-ended data and use it for a real purpose. And in a screener, such usage is seldom warranted. Save people from having to type just because they aren’t going to qualify.

[QUALIFY]

While some of the above is simple, it’s important not to overlook or underestimate the importance of a good screener. It’s not uncommon that so much focus is paid to the core of a survey that the screener doesn’t receive due attention. Consider this a friendly reminder that what comes first in a survey matters – a lot.

Roddy Knowles serves as Director of Product and Research Methodology at Research Now, where he champions how to (and how not to) leverage technology and embrace innovation in market research.

Facebooktwittergoogle_plusredditpinterestlinkedinmail

Leave a Reply

Your email address will not be published. Required fields are marked *