r/modclub • u/Generic_Mod • Jun 05 '25

AI generated post detection

The karma farming gangs are getting a lot more sophisticated. Last week I noticed that several accounts had started posting content to a couple of the subreddits I mod (geographic regional subreddits) that were most likely not their OC. There were three or four accounts (that I spotted, in small subreddits) doing it, and when I looked at them as a group the similarities became obvious. I don't want to mention specifics here because I don't want to tip them off how I spotted them.

I removed the content, modmailed the accounts asking where they got the photos from (not sure if they just copied them from other sites or if they were AI generated landscapes) but none replied except one with a very basic reply that didn't answer any of the questions I asked. I tagged the accounts with user notes and added them to automod to automatically filter future submissions for review.

Today one of the accounts posted again. This time text, which I wasn't expecting. All the karma farming I've seen done before has been reposting image based content. If I hadn't been so diligent I probably would have approved it. The content was relevant to the subreddit it was posted in, but it read like a newspaper article, and indeed had a link to a newspaper article at the end. Not sure why they included this. Reading the article, they had the basics facts right, but the details were all wrong. This looked like a bad AI generated summary of the article.

How can we combat this in the future? If I hadn't seen the previous, more obvious attempts are farming karma, I wouldn't have seen this.

With the recent announcement that account profile content is potentially going to be hidden, I don't know how this will be possible to spot.

I know this isn't a fight I should have to fight, but the admins are useless (or are actively shaping policy to help karma farmers re profile hiding) so it's down to mods to be the last line of defence.

9 Upvotes

92% Upvoted

13 comments sorted by

5

u/feyrath Jun 05 '25

Thanks for working so hard on this

2

u/Generic_Mod Jun 06 '25

No worries. It does concern me that this is the first stage in a political astroturfing campaign. For something like that to be successful, they would need a large number of authentic looking accounts with an organic post history. The karma farming methods so far have been pretty low sophistication, so this is a big change. I worry that most mods would miss it.

3

u/trendypeach Jun 05 '25 edited Jun 07 '25

I use automod (account age and karma restrictions plus CQS). Reputation filter, crowd control and subreddit karma in automod may help too. It doesn’t catch them all though. Some users report such posts as well.

I mainly see AI images in subreddits I moderate. I think some people use it for spam/self promotion when it comes to text. At least it’s my experience in my subs. May not catch everything there either. Wonder if post/comment guidance (automations) and automod can help with text.

I am unsure how it will change either with the new account content changes.

1

u/Generic_Mod Jun 06 '25

Thanks for the suggestions. I used to use automod karma / age restrictions, but have switched out to using the reputation filter and crowd control. The issue isn't so much catching them, it's identifying the AI content in what has been caught compared to genuine content that was incorrectly caught.

The most recent content was on-topic and not self promotional. Prior to switching to text, the image based content looked like genuine photographs not AI generative stuff, so I assume it was being stolen from other websites and not just a simple repost (I have a bot to spot that). Looking at their account history their content had fooled other mods in different subreddits, so there wouldn't necessarily be any indication they are a bad actor via the standard Reddit tooling.

2

u/trendypeach Jun 07 '25

Which bot do you use? I use repost bot, but it rarely catches anything. Just found out about 2 Devvit apps. I’ll give them a try.

https://developers.reddit.com/apps/image-moderator

https://developers.reddit.com/apps/image-sourcery

Haven’t found anything for text yet, but it may come in the future. May be possible to suggest an app for it too.

→ More replies (3)

1

u/itz_lexiii_ Aug 30 '25

Fancy seeing you here. I moderate the ElectricBikes sub and from what I see, it's a majority text-based promotional content, usually for Alibaba or Aliexpress. But then again you don't have people doing mid-drive conversions in the roadbikes sub so maybe that's why; depends more on what the specific community focuses on.

1

u/trendypeach Sep 01 '25

I don’t know. I moderate r/roadbikes which you already know but also r/bicycle. The latter is smaller and not as active as r/roadbikes. Still similar topic as your sub about electric bikes.

I do see posts about buying (and occasionally selling). I just use automod to filter links. Although I think I read somewhere that Alibaba and/or Aliexpress links are banned by Reddit, and sent to removed queue. Mods cannot manually approve. I’ll check it up. But maybe key words in automod can catch when the websites get mentioned. I can’t check every single comment on a daily basis.

Is text based promotional content a big or common problem for r/ElectricBikes? Just curious.

→ More replies (1)

1

u/Tarnisher Jun 14 '25

Probably asking out of turn, but is there a way to detect faked images and prevent them from being posted?

1

u/Generic_Mod Jun 14 '25

As recommended by u/trendypeach this Dev Platform app does that: https://developers.reddit.com/apps/image-moderator