What's a screener?
A screener is a short set of questions used to determine whether someone is a good fit for a research study. You can often include them in a web form or email survey that you use to ask for participants.
Screening questions focus on if someone's experiences make them a good fit for your study. You'll avoid asking for opinions or predictions.
A screener is not a research tool. It does not test ideas or collect findings. Its purpose is to make sure your participants are a good fit for the product, service, or scenario you're testing.
Define who you want to speak with (and who you don't)
Before you write anything, identify who you want to speak with and who you don't. Researchers sometimes call these inclusion and exclusion criteria. For example:
- Relevant experience. People's jobs, levels of education, and life experiences may give them special insights into your research focus. Decide if a set of insights might be what you want or hurt your study. For example, you might not want to talk to renters if you're testing a service aimed at homeowners.
- Demographics and social context. Factors like income level, race, gender, and family size will shape people's perspective. Those perspectives will shape your data. Try to select for demographic criteria that will help you get the answers you need. Usually, it's good to recruit a wide range of experiences. However, you can't recruit every possible demographic. Focus on the demographic criteria that help you learn what you need to learn.
- If they've participated in previous studies. Avoid if possible. You should have a record of who your team has worked with in past research.
- Specialized experience and subject matter expertise. You'll often want to avoid people with expertise in the domain you're testing in. For example, if you're testing whether people can fill out a complex legal form, you may want to avoid including lawyers. Sometimes, the opposite is true. If you're testing a tool for lawyers, you'll look for people with legal backgrounds.
Broader criteria for exploratory research, narrower for evaluative research
Exploratory research helps you build broad, general knowledge about something. Evaluative research helps you learn if something (product, feature, etc.) meets the needs of its audience. In general:
- Exploratory research works best with broader criteria. You'll learn from a range of people and identify emerging patterns.
- Evaluative research often needs narrower criteria to understand a specific audience's experiences.
Use information you already have to save space
Shorter screeners get more responses. You can keep screeners shorter if you make use of what you already know about the people filling it out.
For example, imagine you're recruiting from a user panel or newsletter list. When people signed up for these, they may have provided basic information: Occupation, age, location, what task they were trying to accomplish when they signed up, etc. Pre-filter these lists to include only people who might meet your criteria, e.g. "only people on my list who are at least 60 years old." That way, you don't need to add a question about age to your screener.
Writing questions
The point of a screening question is to help you decide if someone is a good fit for your study. Questions that don't do this don't belong in your screener. Good questions tend to be short, clear, and about life experiences (not opinions or predictions).
Keep it short
Most screeners only need 5-10 questions. People should be able to finish them in a few minutes. Longer screeners may make recruiting more tedious, since people may give up.
Focus on what people have actually done
Avoid asking what they might do or how they might feel. For example:
- Ask like this: “Have you renewed a license or ID online in the past 12 months?”
- Not like this: “Would you use the RMV online renewal tool?”
Questions should have a specific purpose
For each question, identify one piece of information about someone you need. Be careful not to ask questions that gather 2 separate pieces of information. For example:
- Ask: “Have you renewed MassHealth coverage online in the past year?”
- Not: “How confident were you when renewing your MassHealth coverage?”
The "not" example combines asking if someone renewed with level of confidence they felt doing it.
Write close-ended questions
Yes/no questions, multiple choice, and checkboxes are faster to answer. They're also much easier for you to use when reviewing completed screeners.
Avoid opinion and hypothetical questions
Screeners aren't for collecting feedback or testing ideas. In addition, people are often incorrect when you ask them to predict what they would do in a situation. For example, avoid asking things like:
- “Did you like using the application?”
- “How satisfied were you with your last RMV visit?”
- “Would you be interested in trying a new version?”
Test the screener before sending it out
Have a teammate (or 2) run through it and submit it. Ask them which questions were unclear. Also, look for opportunities to remove questions.
You can also try reading it aloud and imagine answering it yourself. This can help you identify where you need to revise.
Data handling and privacy
Screeners often contain people's personal information (PI). Store submissions in a folder where only the people who need to can access them. Ask your organization's legal team what you need to do to protect this information. They will likely review your screener and your process for storing submitted surveys. You might need to document who has access to completed screeners after you submit them.
In general:
- Collect the minimum information required for eligibility
- Avoid sensitive questions unless they’re essential
- Use only approved tools to store and review screener responses
- Limit access to people directly involved in recruitment
- Follow rules for retaining, deleting, and archiving data you collect
Example screeners
- Preparing for Massachusetts Health Exchange and MyMassGov integration. Exploratory research to learn what kinds of challenges people might face when asked to link their Health Exchange and MyMassGov accounts. Identifying these challenges ahead of time allowed product teams to address frictions before they went live.