How to Prevent Cheating on Your Surveys + Best Practices and Examples

So you’ve sent out a survey and now you’ve collected participant responses. Time to start analyzing and reporting on your data, right? Not so fast! It’s best to take a closer look at whether someone or something contaminated your data. While the vast majority of respondents take surveys in good faith, there will always be some rotten apples, also known as “trolls”. The anonymity of online environments can provide a safe place for trolls to express themselves, spread hate, and destroy what others are trying hard to build or improve. In fact, some professionals estimate that up to one-third of online survey respondents these days are not real respondents. They may not even be real humans, but robots or spambots. But not all hope is lost! So, let’s look at how to prevent cheating on your survey.

This article is divided into two parts, to prevent cheaters you can 1: Use what our technology already offers, and 2: Apply these best practices while creating surveys.

Create your free survey, form, or poll now!

1: Use What Our Technology Already Offers

SurveyLegend offers a series of nifty features that can help you find cheaters or block spam robots, or spambots.

• • Built-in Spambot Protection

    Spambots are small computer programs that are written to quickly do boring and monotonous tasks which would take a lot of time for humans to do. So, theoretically, a programmer with bad intentions is able to write some code to fake a respondent who is participating in a survey. Then the bot can open the survey several times and vote in a way that can impact the results.

    However, we have made an anti-spambot system that observes and measures many parameters to see whether a robot is taking the survey or a real person. When our mitigation system detects a spambot, it will silently ignore their votes, so nothing will be collected or registered in our database. However, for the bot, it appears that their responses are registered. Additionally, if our anti-spam system suspects that a respondent may be a bot but cannot judge with 100% certainty, it will instead mark their responses in the exported data as “suspected spambot”. This way users can decide whether to include the data or filter it out. Further, this system automatically goes through old collected data and removes any eventual spambot responses.

  • Please note that we cannot reveal how our bot mitigation system works, as it can make it easier to bot programmers to find ways to cheat more efficiently. Anyways, we will keep improving our spambot protection code, and constantly make it smarter to mitigate more types of spambots.

• • Enable the IP Tracking System

    To help yourself with flagging and cutting specific respondents for quality problems, you may want to take a look at respondents’ IP addresses. You can enable the IP tracking in Configure step, and you can read more about collecting IP addresses from your survey respondents here. IP address represents the network via which a respondent is opening your survey. It could be for example a WiFi network at somebody’s house, or a shared internet network at a public library, or a student accommodation.

    If you suspect that there are cheaters in your audience, you will probably see many of them coming from similar IP addresses. Repetitive IPs may mean that there is one person (or several persons) sitting in the same place, reloading your survey and voting over and over again. But keep in mind that having a few repetitive IPs in the list may simply mean that there are simply a few different (and real) individuals voting, and they are just all using a shared internet network, for example, their WiFi at home. If you want to go even deeper with your analysis, you can use tools that verify whether an IP is blacklisted in any anti-spam databases, such as What Is My IP Address: Blacklist Check.

    It’s important to note that IP addresses are considered as “personal information” by the European GDPR law. Therefore, collecting and storing people’s IPs without their consent is not wise. Please read this article to learn more about how you do it the right way.

• • Disable ‘Multiple Participation’

    All questionnaires made on our platform have a feature that blocks people from submitting surveys more than once. The setting that controls this behavior is located in the ‘configure’ step of each survey and is called ‘Allow multiple participations‘, which is OFF by default.

    Make sure this setting is OFF, but also remember that this feature is only capable of preventing normal internet users from re-opening your survey and polluting your data. This is because the technology that our system uses to know whether the same person is trying to participate in the same survey again relies on ‘cookies’.

  • Due to legal restrictions, we cannot track people and save their data without their consent. Therefore, what we do is to put a small cookie file in their browser’s cache memory. The file does not contain any crazy methods to identify and track a person or match them against a secret database to “prevent cheating”. If we did that, that’d be quite illegal. So the file will only be able to tell our system that this browser has previously opened the same survey link, and that’s it. Therefore, our system can redirect them to another page instead of opening the survey.

    This means if a respondent can re-submit the survey if they:

    • Open the survey again in another browser (which has a different cache memory)
    • Surf the internet in private browsing mode (which clears the cache memory and cookie files as soon as you close a web page)
    • Clear their internet history and start again

    However, since many people won’t have this technical knowledge, this feature protects you from rookie cheaters quite well!

• • Review Time Stamps

    One of the pieces of information that our system captures is the time that each respondent spends on a survey. This is the time it takes for them to open the survey until they either ‘close’ it or ‘submit’ it. The ‘Time spent’ information is located in Individual responses view, under the Progress tab. You can also find this information in the exported raw data. You can give your survey to a few respondents and ask them to finish the survey while observing how long, on average, it takes for them to read the questions, comprehend them, and respond to them. This will give you a good idea about the average time spent on this specific survey (for a specific audience).

    When reviewing your data, if you notice that there are some very long participation times (which are usually infrequent) it should be okay. This usually means that the respondents got perhaps interrupted and resumed the survey later on. However, very short times which are notably below the average are not okay. Most likely, that means that somebody rushed and clicked through the answers, without really reading the questions. That may mean that their responses are really low quality or that they are just cheating.

    Additionally, having multiple similar time stamps which have an increasing sequential ‘start time’ and ‘end time’ can also reveal attempts from spambots (and often submitted/completed surveys).

• Review Device Information

  Another type of information that SurveyLegend collects from respondents is about their devices. Under the Metrics tab in Individual responses, you will find information regarding respondent’s Browser model and version, as well as their OS (operating system).

  Exactly similar responses which are coming from the same device and browser should set off the alarm for possible fraud attempts.

2: Apply These Survey Best Practices

Technology helps you to prevent cheating on surveys, but it can only do so much. So, here’s a look at some other things you can do to prevent cheating on surveys and dissuade frauders who may try to skew your survey results.

• • Use Simple Psychology Tricks

    Simply said, right at the beginning scare off the cheaters, make them understand that their cheating efforts are worthless, because you have means of monitoring the participation process which can easily flag cheaters. This is still useful, even if you do not have any decent way of monitoring responses.

    You as the survey creator can ask respondents to provide a valid email address to be able to continue. Simply put the email question first (which automatically validates emails to some extent), then make it required, and add a Page break after that. This way, respondents cannot continue without providing an email address.

    *To make sure NO cheating happens and everyone has a fair chance of winning in the competition, please type your valid email address for the vote you’re proving to be counted as a valid vote.

    NOTE: This survey is protected with SurveyLegend’s spam-bot mitigation system. Additionally, if we notice a human patterns of cheating such as invalid emails or sudden spikes in votes coming from the same IP, location, or device, we will disqualify both such votes and the competitor who is competing in this contest.

    Have a look at this similar live example.

• • Get a Commitment from Respondents

    Sometimes ‘cheating’ is about providing answers which are not based on honest opinions or personal knowledge or information from respondents. How do you obtain a commitment from respondents not to cheat? According to this research, asking a question like the one below, where you make them commit for a specific positive action (clicking yes) does the trick. Answering such a question constitutes a specific behavioral commitment, which actually works.

    *It is important to us that you do NOT use outside sources like the Internet to search for the correct answer. Will you answer the following questions without help from outside sources?

    • Yes
    • No

    Additionally, some studies suggest that you can easily reduce the number of people who are gaming your survey, by adding “warnings” in questions like this substantially improved data quality.

    *We check responses carefully in order to make sure that people have read the instructions for the task and responded carefully. We try to only use data from participants who clearly demonstrate that they have read and understood the survey. Again, there will be some very simple questions in what follows that test whether you are reading the instructions. If you get these wrong, we may not be able to use your data. Do you understand?

    • Yes
    • No

• • Add Extra Validations Using Logic Flows

    You can validate responses that are typed by your respondents, easily using our survey logic flows. For instance, in the previous example where you ask for email addresses, you can exclude invalid email providers which can generate fake and temporary inboxes and email addresses to users. Some examples of such providers are: Fake Email, Fake Mail Generator, Mailinator, see more examples here.

    Such fake providers can be filtered out if you make a logic rule like this: if > answer to the email question > contains > [@mmailinator.com] or [@email-fake.com], [etc…] > then > skip to > thank you page. Read more about adding logic for text-based questions here.

    You can alternatively only allow certain email addresses to be accepted as valid emails using logic flow. For example, you can only allow emails from your company’s domain to be able to enter the survey by making a logic flow like this: if > answer to the email question > contains > [@my-company-site.com] > then > show > page 2.

    Adding logic validations depends on your survey case. You can come up with other validations and build more elaborate screening paths. For example, you can create questions for multiple responses that will conflict with each other. This way you can detect if a respondent is rushing through the survey, answering randomly, or if they are selecting many options just to get in. If your criterion meets, the logic flows won’t let them in. You will see your qualifying incidence drop dramatically.

• • Avoid ‘River Sampling’ and Go for ‘Panel Sampling’

    River sampling is a technique that mass-invites people to engage with a survey. In this method, researchers use online banners, ads, promotions, offers, and social media invitations. They try to catch the attention of potential respondents, make them click on the link, and hopefully take the survey. Surveyors have no clear idea who the respondents will be, or what psychographic features or demographics will they represent. And usually following up with them again after survey completion is not a part of their plan.

    Panel sampling, on the other hand, is about inviting respondents from an affiliate site. Upon registration, members (aka “panelists”) of such sites are asked to confirm that they are interested in participating in multiple surveys. Researchers then invite specific panel participants by email, based on their qualifying demographic and psychographic characteristics. Members of the panel list are easily trackable and researchers can reach out to them again at any time. Panelists generally receive compensation for their time and efforts.

    While some types of research may work better with river sampling, this is not always the best way of doing research. Many market researchers for instance use river sampling, but it is only as a complement to their ongoing panel sampling. Panel sampling gives a more dedicated and reliable data source. However, sometimes this too carefully selected source of information may result in rigid or over-processed data too.

    But generally, by using the river sampling method, businesses are not able to pinpoint the exact characteristics of their respondents or know who they really are. Because unlike panels, there is no double opt-in verification to make sure that the survey invitees are real, individual people. Anyone with any intention may come in.

• • Include a Minimum of One Open-Ended Question

    Including an open-ended question is always a good idea to prevent cheating on surveys. The best place to add such a question may be at the end of the survey. Make it a ‘required’ question for everyone, and make sure that all respondents will be able to answer it thoughtfully. When analyzing the results, review every response to evaluate its relevance. Bad respondents always give you bad, short, or silly answers. Sometimes it’s just random letters and meaningless words. Other times, they may type in completely generic responses with no elaboration.

    You can give this question a minimum acceptable range to make respondents type at least a certain quantity of characters or words. Later under ‘individual responses’ view, you can easily view and disqualify such respondents.

• • Add a Simple ‘Attention Test’

    You can add a simple question, where you ask respondents to choose a certain choice from a list of provided choices. To do so, you can add a multiple selection questions or a single selection one. Then, in the question, ask them to either choose only one selection or to entirely skip answering this question. If respondents are actually paying attention and if they are not robots, they will do as you ask them in the question.

    How many people live in your household? If you answer this question we will not count your vote, because you are either not paying attention and your responses are therefore not valid, or you are a spam robot.

    • Only me
    • 2
    • 3
    • 4
    • 5 or more

    When analyzing collected data, you can easily filter out respondents who have answered this question, or if their answer is different than what you have asked them to answer. Another idea could be adding logic flows which automatically disqualify the respondents who answer this question, or answer wrong.

Conclusion

No matter what we do, there are always people who cheat or provide dishonest answers. But using a combination of our technologies and these cheat prevention techniques will dramatically reduce the number of cheaters or spam bots in your surveys, polls, and questionnaires. Make sure to at least use a few of these tricks to gather some quality data when you get feedback or do surveys, exams, or online competitions.

Do you find that a lot of people are untruthful on surveys? Has anyone ever tried to skew your survey results maliciously? How do you prevent cheating on surveys? Let us know in the comments!

Create your free survey, form, or poll now!

Frequently Asked Questions (FAQs)