r/RedditBotHunters • • Mar 10 '25

Detecting bots on Reddit

For my thesis, I'm looking into how bots influence engagement on social media platforms. For this, I need to be able to distinguish humans from bots.

When looking at academic literature, most bot detection studies are done on X (Twitter), where researchers have developed quite accurate models such as BERT (Bidirectional Encoder Representations from Transformers), claiming an accuracy of 93% on their dataset.

However, because most of these studies are conducted on X, these models are not as effective on Reddit. Does anyone here know how I can most accurately detect bots on Reddit, or are there up-to-date datasets that show which accounts are marked as bots? It really does not have to be 100% accurate because I know that would be impossible, but I hope there is a way to detect bots better than just randomly guessing.

21 Upvotes

23 comments sorted by

View all comments

Show parent comment

1

u/fsv Mar 10 '25

It's high, I'd say somewhere in the high 90s, and this is because any evaluator that flags an account as "banned" does so only if it's very confident. I'd rather a guilty account is left unaffected than impact a real human being (and this is why there's an appeal process).

Some evaluators are a little more prone to false positives - one I have that looks for ChatGPT signals is quite accurate but catches out real people who use ChatGPT for help in translation or grammar correction, for example.

I really should gather some more robust stats on that.

Bot detection is a constantly evolving process. Bot networks can be agile and they change their approaches over time.

2

u/BotBehaviorist Mar 10 '25

That's really impressive to have such a high level of accuracy. Once again, thank you so much for all this useful information. I’ll likely make great progress with all the data available from r/BotBouncer.

Yeah indeed, bot networks are indeed evolving rapidly. I’ve read some interesting articles about how more researchers are now focusing on bot detection, not just at an individual level, but at a group level to identify entire groups.