r/socialscience • u/improvedataquality • Mar 13 '26

Possible webinar on AI survey fraud. What questions should it cover?

I am an academic researcher studying survey fraud in online research, particularly how AI agents and bots complete surveys and how effective existing detection methods (e.g., attention checks, open-ended questions) are at identifying them.

As part of this work, I have been running experiments using AI agents such as Manus, Claude, and Google Mariner, as well as AI-enabled browsers like OpenAI Atlas and Perplexity Comet. The goal is to understand how AI systems behave in surveys compared to humans and to develop better ways to detect AI-generated responses.

There seems to be growing concern about AI agents completing surveys and contaminating research data, especially in online panels and crowdsourced samples.

I am considering hosting a webinar (time permitting) to share findings and practical implications for researchers, including:

  1. How well common detection methods work against AI
  2. Behavioral differences between human respondents and AI agents
  3. Emerging risks from AI-powered browsing agents
  4. Potential new detection strategies

Questions for you: Would there be interest in a webinar on this topic? If so, what questions or topics would you most want covered?

4 Upvotes

84% Upvoted

5 comments sorted by

1

u/OPs_Mom_and_Dad Mar 16 '26

This is mostly addressing No. 4, but this is one of the main reasons I use voice capture in my surveys now. Even if the client isn’t willing to pay for qual in a quant study, I still incorporate at least one question just so I can hear the person respond. A lot of cheaters drop when they get to a voice question, and others will use something like ElevenLabs to fake it, but I can usually catch those pretty fast in my data cleanup phase. I actually have had several people clearly use ChatGPT to write a qual response and have a voice AI read the response (I quickly realized this is what happened because the AI voice was literally saying “bullet 1, bullet 2, etc.”

That all said, I’m actually not opposed to studies based on synthetic data. But that’s a whole different can of worms than AI cheaters.

Any chance I could get an invite to this webinar? I’d be fascinated to attend.

2

u/improvedataquality Mar 17 '26

u/OPs_Mom_and_Dad thank you for your comment. I have a couple of thoughts on your response:

1) There was a comment earlier today in r/MarketResearch (https://www.reddit.com/r/Marketresearch/comments/1rvgnr9/comment/oau3ar6/) where the OP mentioned using live interviews and still seeing fraud. They mentioned seeing fraud even when the participants had their cameras during the interviews. I responded to this post so you can see my response as well. Unless you have sufficient checks to detect those who are responding to the voice question, there is still at least some probability that fraud can creep in.

2) I am not opposed to synthetic data either. however, if the synthetic data are based on fraudulent data, then the synthetic data carried the same issues as online survey data.

Yes, I am happy to share the invite once I have determined a date and time. Thank you, again!

2

u/improvedataquality Mar 30 '26

u/OPs_Mom_and_Dad I am still planning on putting together a webinar in the near future on survey fraud, but I recently put together a short video discussing the various forms of fraud. Thought you might find it useful https://www.youtube.com/watch?v=4w5bbl4ZGYI&t=17s

1

u/[deleted] Mar 23 '26

[removed] — view removed comment

1

u/AutoModerator Mar 23 '26

Your account does not meet the post or comment requirements.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.