What are screening questions?
In addition to demographic characteristics (age, gender, place of residence, language), you might have other requirements to filter testers.
Screening questions are about adding an extra layer of filtering who gets invited, unlike a survey, which aims to categorize invited participants and can be done as part of the test.
With screening questions, you could filter for example:
Interests and hobbies: mountaineering, art house films, classical music, vegan diet, etc.
Habits: visits to the theater, means of transport, sports, media usage, etc.
Attitudes: politics, religion, behaviors, design preferences, etc.
Usage or visiting of certain websites, apps, or services: Spotify, Netflix, Financial Times, etc.
As you might guess, it is possible to end up with very few suitable testers, or none at all. Because of this, try to create a criteria that matches at least 1 out of 10 testers.
Advantages
Just like demographics, screening questions allow you to only invite specific testers to your test.
You can set up much more extensive and complex criteria and therefore target a very specific subset of testers from our pool.
Disadvantages
The more testers you exclude from the test, the longer you will have to wait for results.
When done wrong, you might end up with too few suitable testers or none at all.
Setting up good screening questions takes time.
Testing with a homogenous group of testers will give you less different perspectives.
So before you set up screening questions, make sure that you actually need them. Always start with demographics and see if you need to further narrow them down.
How to add screening questions on Userbrain
When ordering testers, you can set up screening questions at no charge:
Choose „Single Select“ or „Multiple Select“
Question types
Userbrain offers two different types of questions. If you are new to screening questions, try stick to the Single Choice option.
Single Select
Testers can only select one option per question:
Multiple Select
Testers can select one or more options per question.
Answer validation
In order to define which answers pass the screening process and which ones don't, Userbrain offers three types of answer validation.
Must Select
This option must be selected, in order to pass the screening question.
If you have a Single Select question with more than one answer marked 'Must Select', testers who check any them will pass.
In contrast, Multiple Select questions require all of them to be checked.
Reject Tester
If this option is selected, the tester fails the screening process.
Ignore Answer (only for Multiple Select)
This option allows you to ignore the answer. You might ask yourself why you would ever need that:
Disguise the correct answer. Sometimes it makes sense to add one or two extra options in order to provide a more realistic spectrum. Be careful not to use too many.
Create disqualifying questions, such as "either option A or option B, just make sure it's not C". To do this, mark both A and B with 'Ignore Answer' and C as 'Reject'
Tips for successful screening questions
Let's look at some dos and don'ts to help you write coherent questions that avoid you from ending up with no participants.
Avoid "yes" or "no" answer choices
Testers are eager to test and if you ask them a yes/no type question, their answer might be biased towards a positive response. To eliminate this factor and receive honest responses and well-suited participants, provide multiple select choices instead.
Limit answer choices
Do not provide too many answering options and make sure you have a good balance of qualifying (Approve/Must Select) and disqualifying (Reject Tester) responses. If you are off-balance, your screening process might become very hard to pass. This would make it take much longer and - in extreme cases - have higher charges.
Provide a "none of the above" option
Now that you have limited your answer choices, a 'none of the above' type option (eg. 'Other', 'I don't know', …) is the easiest way to still be inclusive towards every participant. If such an option is not provided, testers might pick an answer randomly, possibly ending up in your study accidentally.
Examples of screening questions
Recruit only testers who own a car
Question: Which of these vehicles do you own?
Answers (multiple select):
Bike (Ignore Answer)
Motorbike (Ignore Answer)
Car (Approve)
None of the above (Reject)
Recruit only testers who mainly use a car to get around
Question: What is your preferred means of transport?
Answers (single select):
Bike (Reject)
Motorbike (Reject)
Car (Approve)
Other (Reject)
Exclude testers who own a snake
Question: Which of these pets do you own?
Answers (multiple select):
Cat (Ignore Answer)
Dog (Ignore Answer)
Snake (Reject)
None of the above (Ignore Answer)
Frequently Asked Questions
What is the purpose of screening questions on Userbrain?
Screening questions allow you to further filter testers beyond demographics. They help ensure that only testers who match specific behaviors, interests, attitudes, or product usage criteria are invited to your test
Do I always need screening questions?
No. In many cases, demographic filters are sufficient. Screening questions should only be used when you need a very specific subset of testers. Overusing them can significantly slow down recruitment.
What is the recommended order when narrowing down testers?
Always start with demographics first. Only add screening questions if demographics alone do not sufficiently define your target audience.
How many screening questions can I add to a test?
You can add up to two screening questions when ordering testers. Because this limit is intentionally low, each question should be carefully designed to avoid unnecessarily excluding testers.
Why is my test taking longer to fill when I use screening questions?
Screening questions reduce the number of eligible testers. The more restrictive your criteria, the longer it will take to find suitable participants.
Is there a recommended success rate for screening questions?
Yes. As a rule of thumb, your criteria should match at least 1 out of 10 testers from the pool to ensure reasonable turnaround times.
Can screening questions reduce the diversity of feedback?
Yes. Highly specific screening criteria can result in a very homogenous tester group, which may limit the range of perspectives in your results.
How do I know testers answer screening questions honestly?
All Userbrain testers are required to agree to our Terms and Conditions, which explicitly state that they must provide correct and honest information at all times, including when answering screening questions.
Testers are aware that answering screening questions untruthfully can result in removal from the Userbrain platform.
These consequences create a strong incentive to answer screeners accurately rather than guessing to qualify.
If you believe a tester did not provide truthful information in a screening question, please use the “Report a problem” function on the test recording. Our team will review the case, and if the report is valid, the session will be refunded within 24 hours.
What happens from the tester’s perspective when they fail a screening question?
If a tester fails a screening question, they are simply not invited to the test.
There is no penalty for failing a screener, and testers are not told which specific answer caused them to be disqualified. They can continue taking part in other tests on Userbrain as usual.
This approach ensures a neutral tester experience and helps prevent testers from trying to optimize or “game” their answers.
Do failed screening attempts count toward my usage?
No. Testers who fail screening questions do not count toward your tester quota and are not charged. You only pay for testers who successfully pass the screening process and complete your test.
Can I change or remove screening questions in a running test?
Yes. You can stop a running test, update or remove the screening question, and then launch the test again.