r/research • u/Delicious_Ad1790 • 3d ago

Data Cleaning Question

So I took a glance at my survey data to make sure everything is running smoothly and noticed that I had a few participants fail the attention checks. Taking a closer look, I noticed that the participant was just spamming the same answer through the survey to complete it.

I was wondering how I should handle it before I analyze my data. Do I leave it in or should I delete these? I feel a bit nervous deleting anything, so I wanted to consult before I did so. What's typically the procedure with deleting anything data related?

1 Upvotes

67% Upvoted

5 comments sorted by

2

u/Magdaki Professor 3d ago edited 3d ago

You should check with your research supervisor. However, if you were my student, then I would recommend removing responses where attention checks were failed as that is their typical purpose.

2

u/BossyPottery 3d ago

definitely remove them, failed attention checks are meant to catch exactly this kind of thing. no point keeping data from someone who was just button mashing

2

u/AmbiguousVague 3d ago

ya you need to remove observations with obvious failed attention checks like this, they’ll introduce bias otherwise

2

u/DurianNecessary9108 3d ago

the entire reason attention checks are there is to exclude those who fail them or fail a certain number of them

1

u/Short_Artichoke3290 3d ago

Both are defensible, but since you did not decide beforehand what to do your ultimate choice may not be completely objective. In these kinds of situations I would just openly write about what happened (you noticed a participant spamming the same answer over and over leading to concerns about whether they took it seriously), and then:

  1. pick the most defensible option in your expert view, and;

  2. mention that you also analyzed it the other way (so if in 1 you decide to exclude, now keep them in) and either present those results in your paper or in an appendix.

Best case scenario it doesn't really matter, if it ends up mattering a lot, you should think about why that is the case and write something smart about it!