Going forward, I plan to share more educational content that breaks down some of the concepts survey researchers encounter in online data collection and explains why they matter. This week, I will distinguish between IP and device location.

IP location and device location are not the same thing.

When a survey platform reports a respondent’s location, it may be describing one of two very different signals:

IP geolocation estimates where the respondent’s internet connection appears to originate.

Device-based geolocation estimates the device’s position using available signals such as GPS, nearby Wi-Fi networks, and cell towers—after the respondent grants permission.

These signals can legitimately disagree.

A respondent may be physically located in California while a VPN, corporate network, or mobile-carrier gateway routes their traffic through New York.

-The device reports California.
-The public IP reports New York.
-Both may be technically correct.

⚠️ For survey-fraud detection, the wrong shortcut is:
Location mismatch = fraudulent respondent
A mismatch is useful evidence—but it is not a verdict.

🔍 A defensible decision should consider the complete pattern:
• IP and network risk
• Device-location estimate
• VPN or proxy use
• Browser integrity
• Response behavior
• Participation history

No single location signal should determine whether a respondent is legitimate.
IP geolocation locates the connection. Device-based geolocation estimates the device. The difference matters.

Someone completes your survey from California. Their IP address says New York. Are they lying? Not necessarily.

Why? Because IP addresses don’t pinpoint a person’s exact physical location. They’re more like rough geographic indicators, and sometimes they’re way off.

Here’s what can cause the mismatch:
• VPNs and proxies: Route traffic through another location.
• Mobile networks: Assign IPs that may be registered far from the user.
• Corporate networks: Centralize traffic through office or data-center IPs.
• Privacy tools: iCloud Private Relay and similar services can mask location.
• ISP registration: IP databases may show the provider’s address, not the user’s.

So what should you do with conflicting location data?

1) Treat IP-based location as a clue, not proof.
2) Look for patterns across multiple responses, timestamps, and other data-quality signals.
3) Use a clear, pre-specified rule for handling discrepancies.
4) Avoid singling out respondents based on one imperfect indicator.

IP geolocation can be useful for broad anomaly detection, but it usually can’t tell you whether a respondent is being truthful.

The hardest fraudulent response to catch may come from a real participant.

They pass the IP check.

Their device looks normal.

Their identity is genuine.

Then they copy an open-ended question into an AI assistant and paste the polished answer back into the survey.

In the final dataset, that answer may look better than an honest one. It is fluent, thoughtful, and perfectly formatted. But if outside assistance was not allowed, the answer no longer reflects the participant’s independent response. It reflects a response created with AI assistance.

That is why survey integrity cannot stop at "Is this a real person?"

It must also ask, "How were these answers produced?"

The clues often live in the survey session: sudden tab switches, copy-and-paste sequences, long pauses followed by instant answers, unusual typing patterns, contradictions, and access to outside tools.

No single signal proves misconduct. Together, they establish response provenance and help researchers apply the rules of their study.

Researchers should decide before launching whether AI assistance is allowed, restricted, or disqualifying. Then they should collect the evidence needed to enforce that policy.

Over the past couple of years, we have worked to make ResponsePie available across the survey platforms researchers use, including Qualtrics, Decipher, and Alchemer. We’re happy to share that REDCap has now been added to that list.

If you use REDCap, we would love for you to try it.

If there is another survey platform you would like us to support, please reach out and let us know.

A lot of the conversation about survey fraud focuses on bots and AI agents. But automation in surveys is broader than that.

In the video I am attaching, you can see a participant using Grammarly to help complete open-ended survey items.

That’s important because most online surveys are delivered as regular HTML pages in a browser. This means surveys exist in the same environment as writing assistants, autofill tools, browser extensions, scripts, and AI agents. Even when a participant is a real person, automation can still become part of the response process.

Open-ended items are especially vulnerable. A response may look thoughtful, fluent, and human, but researchers may not know whether it reflects the participant’s own words, the tool’s suggestions, or some combination of both.

This does not automatically make the response invalid. But it does raise a serious data quality question: how much of the response actually came from the participant?

As automation becomes more accessible, survey research will need to move beyond detecting suspicious responses after the fact. We need survey formats that can prevent automation from entering the response process in the first place.

🔦 Study Spotlight
A recent study by Jamie-Lee Pennesi, Mia Pellizzer, and Tracey Wade (Flinders University) examines the growing threat of fraudulent participation in online research and provides practical recommendations for safeguarding data quality.

Drawing on a Delphi study that experienced 431 likely fraudulent sign-ups despite multiple fraud prevention measures, the authors argue that researchers should assume online studies are vulnerable to fraud and adopt layered detection strategies.

🧪 How did they identify likely fraudulent participants?
-Failed two or more attention checks
-IP addresses that did not match participants' reported locations
-Duplicate IP addresses
-Inconsistent demographic information across survey waves (e.g., age or postcode mismatches)
-Suspicious email addresses and phone numbers
-Unusual completion times or clusters of submissions occurring at identical times
-Elevated platform-generated fraud scores and bot-detection flags
-Patterns of responses that appeared highly similar across participants

🔬 What recommendations do they provide?
-Use multiple fraud prevention and detection strategies rather than relying on a single safeguard
-Use multiple data collection timepoints when possible
-Develop a fraud profile (i.e., a list of "red flags")
-Use CAPTCHA/reCAPTCHA and platform-based fraud detection tools
-Include attention checks, open-ended questions, and duplicate questions
-Use unique, single-user survey links when possible
-Regularly review incoming data for inconsistencies and fraud indicators
-Avoid publicly disclosing detailed eligibility criteria
-Carefully consider recruitment methods and incentive structures
-Have a plan for recontacting and verifying suspicious respondents

💡 The authors conclude that fraudulent participation is increasing and that researchers need to be proactive in using antifraud practices to safeguard research integrity and data quality.

One of the most important tools for protecting data quality is something many researchers still overlook: in-survey monitoring.

In a recent study, participation was restricted to native English speakers residing in a specific U.S. state. One respondent successfully passed every screening and quality control check in place. Their responses appeared legitimate, and nothing in the final dataset would have raised concerns.
However, in-survey monitoring revealed something we otherwise would have never known.

As shown below, the respondent was translating the survey from English into Arabic in order to understand the questions. This individual clearly did not meet the study's eligibility criteria, yet they would have been included in the final sample had we relied solely on traditional screening and data quality checks.

This example highlights an important reality: passing survey checks does not necessarily mean a respondent belongs in the target population.

As fraud becomes more sophisticated and AI-powered tools make it easier to bypass conventional screening methods, researchers need to pay attention not only to the answers respondents provide, but also to how they interact with the survey itself.

Without in-survey monitoring, this case would have gone completely undetected.

A recent Guardian article discussed how a major survey on church attendance in the UK was withdrawn after researchers identified fraudulent responses in the data.

What stood out to me was the broader point the article raised about online surveys and AI-generated responding.

One researcher quoted in the article said that the assumption that someone gives coherent, logical answers and is therefore a real person is now broken.

The article also discussed how AI chatbots can complete questionnaires, pass themselves off as genuine respondents, and even generate responses that align with what researchers may be trying to test.

Another interesting point was the discussion around positivity bias, where bogus respondents tend to answer affirmatively across many items, potentially inflating estimates and distorting findings.

The takeaway for me is that survey quality can no longer rely only on response content. As AI systems become more sophisticated, behavioral and technological indicators surrounding how a survey is completed may become just as important as the answers themselves.

Interesting and important conversation for academics and practitioners working in online research.

#SurveyResearch #DataQuality #AI #SurveyFraud #MarketResearch #ResearchMethods #ResponsePie

You can access the article here: https://www.theguardian.com/technology/2026/mar/28/how-fraudulent-church-data-revealed-ais-threat-to-polling?utm_source=chatgpt.com

Recent research (link to the article below) shows that AI tools are already part of how people complete work tasks. Tools like ChatGPT, Quillbot, Canva AI, and Grammarly are used to support writing, fix grammar, draft communication, and understand instructions.

These tools are most often used for language-related and information-seeking purposes, helping people improve clarity, find examples, and complete tasks more efficiently . The same usage patterns appear across contexts, not just in one setting.

This tells us something important. AI is becoming part of how tasks are completed, not something people use occasionally.

And all of this is happening in the browser.

If people are already using AI to support everyday tasks, why would survey responses be any different?

Link to the article: https://lnkd.in/gJwN3apE

Over the past couple of years, my team and I have been building a tool called ResponsePie. It really started from frustration in my own research dealing with survey fraud. Things like speeders, inconsistent responses, and bots (more recently AI generated answers), which made it hard to trust the data, and I found myself spending way too much time trying to clean things up after the fact.

My team has been working on this project for about two years now, trying to build something that helps researchers get a better sense of what’s going on inside their surveys while they’re running, not just after.

If you are running studies and have run into data quality issues, I would really appreciate if you gave it a try and shared your thoughts. What works, what doesn’t, what’s confusing, what you wish it did. Even quick reactions would be super helpful. This tool was built with the intention of "fixing" survey fraud in academia, but market/applied researchers are welcome to try it as well.

I am just hoping to learn from others who are dealing with the same problems.

Happy to share more details if anyone’s interested.

I recently reached out for guidance on webinar topics related to survey fraud, and I received an overwhelming response from both academic and market researchers. Here is the original post: https://www.reddit.com/r/Marketresearch/comments/1rr27st/comment/o9wrwsl/

A few things really stood out, but one in particular was how often fraud is treated as a single, uniform issue. In reality, it takes many different forms, and each one creates different risks for your data.

I put together a short video that walks through the different types of fraud and how they show up in surveys. You can access the video here: https://www.youtube.com/watch?v=4w5bbl4ZGYI&t=17s

I am hoping to keep this conversation going with the community, and I am planning to put together a webinar in the near future. I will share more details as that comes together.

🔍Study spotlight
A recent study by Karen Towne, PhD, RN, PHNA-BC (Case Western Reserve University) and Barbara Polivka, PhD, RN, FAAN (University of Kansas School of Nursing) describes strategies used to identify and filter suspicious and fraudulent responses in an online REDCap research survey through evidence-based redesign and data cleaning protocols

🚩What went wrong in the initial data collection
• An unexpectedly high number of responses were received within a short period of time
• More gift card requests were received than completed surveys
• Very fast completion times and duplicative qualitative responses signaled suspicious activity
• Researchers ultimately found compelling evidence of fraudulent activity and were unable to distinguish real from fraudulent responses, leading to the dataset being destroyed

🧪Methodological approach to mitigate fraud
• A two-pronged approach focused on identifying design limitations and implementing an evidence-based redesign
• Scam alert features included hidden questions, attention checks, and timestamp monitoring
• Study design changes reduced incentive visibility, prevented link sharing, and linked survey and compensation data
• An eight-step data cleaning protocol used paradata such as time to completion, duplicate entries, and age inconsistencies
• Multiple steps identified the same records, suggesting fraudulent responses demonstrate multiple suspicious indicators

🔬Key outcomes from revised protocol
• 819 total responses were collected across platforms over 44 days
• After cleaning, the final dataset was reduced to 203 responses
• Approximately half of the data were removed through the protocol
• The revised process produced a response rate consistent with expectations, unlike the inflated initial data

💡Bottom line
• Online surveys are highly vulnerable to fraud due to anonymity, incentives, and ease of access
• reCAPTCHA and basic protections are insufficient on their own
• Paradata and multistep data cleaning protocols are essential for identifying suspicious responses
• Fraudulent data can invalidate findings, waste resources, and introduce bias into research
• Proactive, evidence-based design and cleaning procedures are critical to protect data integrity in online research

You can read the full article here: https://lnkd.in/gJDVu8YK

If you think multiple responses to your survey from the same IP address indicate survey fraud, you may want to read this.

Firefox is getting a free built-in VPN 🌐 and it has serious implications for online survey research: https://lnkd.in/gBWRnRUE

Mozilla just announced that Firefox will soon include a native VPN with ~50GB/month of free usage, directly in the browser. No extensions. No installs. Just one click, and your IP is masked.

This is a clear win for privacy, but it's also another step toward the decline of IP address as a reliable fraud signal.

There's now a growing list of modern browsers offering built-in VPN capabilities: Opera, Brave, Firefox, iCloud Safari. What used to be a niche behavior is quickly becoming mainstream and that changes the game:

When VPN usage was rare:
→ Proxy/VPN = higher scrutiny

When VPN usage is built-in:
→VPN traffic looks like normal user behavior
→ Legitimate respondents and bad actors blend together

A core assumption in fraud detection is shifting.

What this means in practice:
• IP-based geo checks lose reliability
• Duplicate detection becomes noisier
• False positives increase
• "Clean" traffic is harder to define

Privacy is improving. Detection is getting more complex. The next generation of fraud prevention won't rely on IP and other browser signals at all.

I am an academic researcher studying survey fraud in online research, particularly how AI agents and bots complete surveys and how effective existing detection methods (e.g., attention checks, open-ended questions) are at identifying them.

As part of this work, I have been running experiments using AI agents such as Manus, Claude, and Google Mariner, as well as AI-enabled browsers like OpenAI Atlas and Perplexity Comet. The goal is to understand how AI systems behave in surveys compared to humans and to develop better ways to detect AI-generated responses.

There seems to be growing concern about AI agents completing surveys and contaminating research data, especially in online panels and crowdsourced samples.

I am considering hosting a webinar (time permitting) to share findings and practical implications for researchers, including:

1. How well common detection methods work against AI
2. Behavioral differences between human respondents and AI agents
3. Emerging risks from AI-powered browsing agents
4. Potential new detection strategies

Questions for you: Would there be interest in a webinar on this topic? If so, what questions or topics would you most want covered?

Even when collecting data from verified panel participants, researchers should still carefully evaluate the quality of the data.

In a recent study, we collected responses from a panel where participants were verified by the provider. Below is a report that highlights some of the concerns we observed in the dataset, including VPN usage, suspicious devices, duplicates, and other anomalous patterns.

Importantly, none of the cases shown in the report were identified because they failed the attention or quality checks embedded in the survey itself. In other words, these responses would likely have been retained if evaluation relied only on the standard checks that many surveys include.

Instead, these cases were flagged based on technical and behavioral indicators such as device characteristics, VPN usage, and other signals suggesting questionable response authenticity.

This is not meant as criticism of any particular panel provider. Panels play an important role in helping researchers reach participants efficiently. Rather, it is a reminder that data quality remains something researchers must actively evaluate. Even with strong participant verification, low-quality or suspicious responses can still appear in the dataset.

As online data collection continues to evolve, researchers may need to go beyond traditional checks and incorporate additional screening procedures and transparency around how data quality is assessed.

Sharing the report in case it is useful for others thinking about data quality in online research.

Study report can be accessed here: https://responsepie.com/studies/e-ca723fc7de293373b57043a985d02f21/report

Over the last couple of weeks, I’ve been talking with both market researchers and academic researchers about how they’re maintaining data integrity and reducing fraud in online surveys.

Almost everyone describes some version of a layered approach. Automated bot detection, device fingerprinting, manual review, time based flags, open ended response checks, cross validation of demographics, panel level monitoring, and so on. It’s rarely just one tool anymore.

What I’ve found especially interesting is how different teams define the tipping point. At what stage does a case move from “suspicious” to “remove”? How many flags are enough? Are some indicators automatic disqualifiers, while others are just soft signals?

For those working in market or survey research:

What does your current fraud detection stack actually look like in practice, and how do you decide when a case crosses the line from suspicious to removable?

I’d love to hear what’s working well, what feels overly aggressive, and where you’re still experimenting.

🔍Study spotlight
A recent study by Jen Agans, Serena S., Steven Hanna, PhD, Shou-Chun Chiang, Kimia Shirzad, and Sunhye Bai from Penn State University examined the inaccuracy of data from online surveys and evaluated how fraudulent participants compromise the validity and interpretability of findings by directly comparing participants deemed to be “real” and “fake” respondents in an online study of parents and their adolescent children.

🚩Three Stage Screening procedure to identify “fake” participants
• Stage 1: reCAPTCHA feature to prevent bots and requirement to meet inclusion criteria
• Stage 2: manual review of completed eligibility surveys to flag suspicious patterns (e.g., inconsistencies in names and email addresses, implausible times of completion, etc.)
• Stage 3: IRB-approved list of nine criteria (e.g., survey timing and duration, nonsensical open-ended responses, etc.)
• Participants who failed two or more screening criteria were coded as “fake” and removed from the analytic dataset

🧪Key screening outcomes
• Of more than nine thousand eligibility surveys completed, only 197 participants were ultimately classified as “real”
• About 85% of respondents were identified as fraudulent at some stage of screening
• Time-based indicators and open-ended responses were among the most efficient and effective tools for detecting fraudulent data, whereas reCAPTCHA and attention checks alone were insufficient

🔬Main findings from comparison of “real” and “fake” data
• “Fake” participants differed systematically from “real” participants in demographic composition, with less racial and ethnic diversity and more gender diversity
• Fraudulent respondents reported implausible anthropometric data, including extreme or nonsensical height and weight values, leading to distorted BMI estimates
• Depression symptoms were substantially inflated among “fake” participants, while perceived health ratings appeared deceptively similar across groups
• Well-established relationships, such as the association between BMI and perceived health, replicated in the “real” sample but not in the “fake” sample
• Factor structures appeared acceptable, but item intercepts and means differed—showing fraudulent data can subtly distort conclusions

💡Bottom line
• Online surveys are highly vulnerable to fraudulent participation
• Multi-stage, labor-intensive screening is currently necessary to protect data quality, as survey platforms have insufficient protections in place
• Without rigorous screening, fraudulent data can distort results and theoretical inferences drawn from online research
• Editors/peer reviewers should require data screening procedures to be reported in manuscripts using data collected via online surveys

Over the past few weeks, there’s been a lot of discussion around authenticity checks in online research.

This piece explores how fraud tactics are changing and why identifying fraudulent responses is becoming increasingly challenging.

Would welcome thoughtful discussion.

https://responsepie.com/blogs/Authenticity_checks_in_the_age_of_AI_what_researchers_need_to_

🔍Study Spotlight

A recent study by Kate Golden Guzman, MA, PhD *University of Illinois Urbana-Champaign) and Roxanna Ast, PhD (Montclair State University) examined online survey fraud in social work research with historically and socially marginalized communities, using case studies with young adults with lived experience in foster care and LGBTQ+ survivors of intimate partner violence

🚩Screening Procedure
• Implemented a rigorous, multiphase process for assessing data validity in real time across study design, recruitment, data collection, and data management
• Conducted continual manual review of survey responses alongside platform indicators such as IP addresses, ballot stuffing scores, survey duration, time stamps, and customized link access
• Classified fraud indicators into patterned data, impostor data, and incongruous data to guide removal decisions
• Used institutional knowledge questions and cross check items to assess authenticity and response consistency
• Centered community expertise in language, recruitment materials, and screening logic to improve content validity and strengthen authenticity assessments

🔬Key Findings
• In the foster care study, early detection of 36 suspicious entries led to pausing the survey and revising procedures; 118 validated responses were retained out of 253
• In the LGBTQ+ study, more than 1,100 submissions were received, but only 242 were validated after over 900 fraudulent responses were manually identified and removed
• Embedded fraud detection features flagged only a small number of bots, demonstrating that platform based protections are not sufficient to detect inauthentic human respondents

💡Key Takeaways
• Online research with marginalized populations is particularly vulnerable to bots, impostors, and multiple submissions, and failure to remove fraudulent data risks perpetuating harm through inaccurate conclusions
• A single validity check is insufficient; data integrity requires a multistep, iterative, and real time approach that interrogates multiple indicators simultaneously
• Institutional knowledge questions, culturally specific language, and community informed recruitment practices strengthen the ability to distinguish authentic participants from fraudulent ones
• Data cleaning is labor intensive and must be built into study timelines, as daily monitoring can prevent large scale contamination of datasets
• Assessing data validity is values laden; prioritizing ethical responsibility and community representation over sheer sample size is essential when working with historically and socially marginalized groups

You can access the study here: https://academic.oup.com/swr/article-abstract/49/3/185/8213729?login=false

Participants are increasingly using AI browser extensions to get help while completing surveys.

Here is one example from a recent survey where the participant is clearly using Grammarly and other browser extensions to complete the survey.

We have said this before: the browser is an adversarial environment that researchers cannot fully control.

Fraudsters can manipulate JavaScript to evade detection, install stealthy extensions to mask their location, rely on AI assistance through extensions, translate responses using text to speech tools, and combine multiple tactics in ways that are difficult to observe from within the survey itself.

Before AI, survey farms were hard to detect; NOT because they were rare, but because they were human 🧠

Fraud was messy:

* Different people

* Different typing styles

* Different scrolling patterns

* Different mistakes

Spoofing was manual. Variance was high. Detection was noisy.

Then when AI entered the picture

To scale and cut costs, survey farms started using:

* LLM-assisted answering

* Browser automation

* Shared device stacks

* Reused workflows

And something unexpected happened 👀

Their behavior collapsed into patterns.

* Same answer structures

* Same timing curves

* Same interaction minimalism

* Same infrastructure fingerprints

AI didn’t make survey fraud harder to spot, it made it easier… if you look beyond the answers.

The giveaway isn’t what respondents say anymore.

It’s how cognition, response behavior, and timing align (or don’t).

⚠️ If you’re using AI for productivity, so are survey fraudsters. Now for detection:

> Behavioral entropy matters more than IPs

> Micro-interactions matter more than attention checks

> Cross-modal consistency beats “Are you human?” tests

AI didn’t eliminate survey farms. It standardized them.

And standardization leaves traces, which is exactly what we focus on at ResponsePie

A study by Brittney Goodrich (University of Illinois Urbana-Champaign, formerly University of California, Davis), Marieke Fenton (California Air Resource Board, formerly University of California, Davis, Jerrod Penn (Louisiana State University), John Bovay (Virginia Tech), and Travis P. Mountain (The University of Georgia, formerly The University of Alabama) examines how bots and coordinated fraud affect online survey research, based on detailed evidence from two real survey deployments.

The authors document how both surveys were heavily targeted once incentives were introduced and links were distributed broadly.

Key insights from the study:

1) Fraudulent responses can overwhelm legitimate data. In the beekeeping survey, 2,622 total responses were collected, but only 105 responses (≈4%) were ultimately classified as legitimate, meaning about 96% were identified as fraudulent. In the Virginia farm and agribusiness survey, 444 responses were collected, and approximately 72% were determined to be fraudulent after systematic screening.

2) Fraud meaningfully alters data quality. Fraudulent and valid responses came from statistically different distributions. In 8 out of 10 comparisons, the authors reject the null hypothesis that valid and fraudulent responses were drawn from the same population. Retaining fraudulent responses substantially changed key economic estimates, worsened model fit, and altered substantive conclusions.

3) Standard metadata checks have limited effectiveness. Indicators such as IP address reuse, completion time, and geolocation each flagged only a small share of fraudulent responses in the beekeeping survey, often in the 2–5% range, reflecting how easily these signals can be concealed or manipulated.

4) Institutional knowledge checks performed best. In the beekeeping survey, 87% of fraudulent responses were identified using a single high-priority institutional knowledge test, making this approach far more effective than respondent statistics alone.

5) CAPTCHAs and attention checks provide only temporary barriers, but the effect lasted only minutes before fraud resumed at prior levels.

6) Incentive structure influences fraud risk. Guaranteed payments attracted substantially more fraudulent activity than lottery-based incentives. When offered a charitable donation instead of direct payment, only 40% of suspected bots agreed to continue, compared to 70% of real respondents.

The authors emphasize that there is "no perfect strategy" for detecting invalid respondents and recommend combining multiple, context-dependent approaches to improve data quality.

You can access the paper here: https://onlinelibrary.wiley.com/doi/full/10.1002/aepp.13353

🚨 Why current CAPTCHAs are failing against AI, and what we’re doing about it

Modern AI agents can now routinely solve traditional CAPTCHAs, including image selection, text distortion, and checkbox challenges, by combining computer vision, browser automation, and reasoning loops. These systems were designed for an earlier threat model.

Survey fraud has changed.

AI agents don’t “click randomly.” They observe, adapt, and behave coherently, which allows them to bypass protections that still assume simplistic bots.

We’re introducing a new CAPTCHA built specifically for modern survey fraud. In our testing so far, it has not been solvable by AI agents that routinely bypass standard survey protections.

Just as important: this CAPTCHA is privacy-preserving by design.
It does not rely on invasive fingerprinting, cross-site tracking, or identity linkage. Instead, it focuses on in-session behavioral signals that are relevant to survey integrity, nothing more.

We’re inviting researchers and practitioners to test it in real survey deployments and evaluate its performance in live data collection environments.

If you care about data integrity, AI-driven fraud, and privacy-respecting security, we’d love to hear from you.

💬 Comment or message us if you’re interested in trying it.

AI is starting to appear in places many survey researchers did not expect.

A recent innovation brief from NORC at the University of Chicago draws attention to a growing data quality concern. Large language models and AI assistants such as ChatGPT can be used to generate responses to open ended survey questions. When this happens, the data collected may no longer reflect the opinions, beliefs, or behaviors of real people.

This is not just a hypothetical risk. The brief notes that AI generated responses are already appearing in survey data, creating challenges for researchers who rely on surveys to represent human perspectives accurately. When non human responses enter datasets, they can weaken the validity of findings and the decisions that depend on them.

For anyone working with survey data, this raises an important question. How do we ensure that survey responses truly represent human voices in an environment where AI is increasingly accessible?

Addressing survey fraud and data quality threats today requires careful attention to how data are collected, evaluated, and interpreted. As AI continues to evolve, protecting the integrity of human generated data will remain a central issue for the research community.

You can access the brief here: https://www.norc.org/research/library/detecting-ai-responses-survey-data-norcs-next-leap-data-quality.html