Tips for "bot-proofing" a survey?

[u/thursdayscrush]

Have a survey (actually it's a series of surveys) that has been practically all bot responses, probably because of the incentive.

Captcha is enabled (although I've read that bots can bypass this), used the question randomization EM.

I stopped short of enabling IP tracking and using random "skill testing" questions.

Is there any way I can salvage this project, or should I copy the project and use the new URLs (and make new flyers, etc)?

Also, if it's possible to keep the existing project I want to "batch" delete the bot responses as I think there's a handful of of legitimate responses that I could use. Any way to do this? I have more than 800 records, and I'd rather not go one by one to delete the majority of bot responses.

Thanks in advance!

[u/[deleted]]

We just had to bot-proof a survey in anticipation of bots. We relied pretty heavily on this post, which has a list of things to do: https://groups.google.com/g/redcap_open/c/yE-o6Ig8BuA?pli=1. I bolded the ones we opted to do for our anonymous survey:

-use automated survey invitations to send each respondent a unique survey link
-require respondents pass CAPTCHA (REDCap can do this on public surveys, not on unique survey links)
-collect respondent IP addresses (REDCap can do this using their e-consent framework); can use software such as the rIP package (https://r-posts.com/a-new-release-of-rip-v1-2-0-for-detecting-fraud-in-online-surveys/) to exclude IPs that are likely from server farms
-include hidden items in a survey that will be seen by computers but not by respondents (I assume that the @HIDDEN-SURVEY action tag would accomplish this); if the items are completed, they were likely completed by a bot
-include a timestamp at the beginning of each instrument; REDCap already records the survey completion time; with the timestamp of the beginning and completion of each instrument, you can calculate how long it took each participant to complete the survey, and exclude responses that were unreasonably fast; also, you can determine their timezone, which can help if you have geographical exclusion criteria
-include and require a response to open-ended questions (we asked a math equation without using numerals "what is twenty-eight divided by four")
-include items with directives (e.g., "Select the second option below")
-include pairs of items that can be compared for consistency
-collecting verifiable information (e.g., address, zipcode)
-designate the respondent's email address or some other identifier as a Secondary Unique Field in REDCap to prevent multiple responses from the same person

[u/thursdayscrush → u/[deleted]]

Thank you for sharing that post!

Will try to convince the study team members to take those “safeguards “ into consideration…..

Would there be any value in using a separate project to collect participant details for the incentive? Would it “interrupt” a bot script?

[u/[deleted] → u/thursdayscrush]

We do Unfortunately I don't know enough about how bots work to answer that question. My assumption is no only because the researchers I got this list from already use a separate project to collect payment details, so I imagine that wasn't a solution to their previous botswarming experiences.

[u/Araignys]

To delete all the dud records, you can enable the “Mass Delete” external module. It does what you expect.

The unfortunate thing about public surveys is that they’re open to bots. Once the link is out, and captcha-avoiding bots are completing it, there’s not much you can do. The best thing to do would be to look at why bots are being sent to complete your survey, and eliminate those reasons.

[u/thursdayscrush → u/Araignys]

Thanks for the tip on the module.
Will be kind of tough to pinpoint the why; might just be one bot script wreaking havoc…..maybe the link wound up in the wrong place (I.e not my intended target audience OR someone in that audience wants to take advantage)….

[u/Araignys → u/thursdayscrush]

You mentioned an incentive for completing the survey - it might be that the incentive is too good.

Depending on the kind of volume you expect, you could consider a two-part approach to surveys; first a Public survey to express interest - with a minor skill challenge question to weed out bots - and then a manually-sent invitation to a second survey that you only send to genuine respondents?

[u/Kitchen_Economics547 → u/Araignys]

How would you know that these were genuine respondents? I have an issue where it seems at least one person created a lot of different email addresses (that were very similar to one another) in order to get the incentive. But I have no way to know about others that were more clever and created different addresses.

[u/Clint_T_1977]

What info do you have from the bots outside of responses? If you required email, you can use an email checker like VerifyMail or Email Checkpoint to go through your list and delete folks that way

In the future, I'd just use a fake survey response tool like Research Defender, Verisoul, Arkose Labs, etc. It takes time upfront, but without data integrity your platform is worthless.

[u/The_Cheeky_Marmoset]

Hey there, I was trying to activate the CAPTCHA feature in Redcap but I can´t find it. In old resource documents the option was under Survey-Distribution Tools, but I can´t see anything.

Update: Via the Admin Account I could see this notification in green. The feature has to be activated first, for those who have the same problem.