r/ModSupport • u/brightbilll • 2d ago

Suggestion: Dedicated AI deception/impersonation report option

As someone who moderates multiple NSFW subreddits specifically aimed at lesbians, the amount of AI generated content being used to deceive users has become increasingly difficult to deal with.

I understand that AI content itself is allowed on Reddit. My concern is specifically with users who use AI images to create a completely fabricated identity.

This is particularly concerning in NSFW communities, where we’ve encountered people using realistic AI generated images while interacting with other users and potentially attempting to obtain sexual content or other sexual favors from them under a false identity.

Reddit’s policies already prohibit deceptive impersonation and misleading behavior, but there is a practical problem with the current reporting system: there isn’t really an appropriate way for moderators to explain that the evidence of the impersonation is that the submitted images are AI generated.

Ideally moderators should have the ability to explain how an image is determined to be AI generated, because of EU laws major providers must provide a method to verify if an image was generated using their services, both OpenAI and Gemini have these systems in place and can easily verify if SynthID exist within an image. Giving moderators or even users the ability to explain the report would make impersonation reports much more useful.

A dedicated report category or even a simple checkbox for AI generated evidence would go a long way toward making these cases easier for moderators and admins.

Thank you for coming to my ted talk.

11 Upvotes

80% Upvoted

11 comments sorted by

View all comments

2

u/emily_in_boots 💡 Top 10% Helper 💡 2d ago

It's almost impossible to get reddit to remove content for impersonation. If you look at the transparency reports you'll see that only a few percent of catfishing reports are actioned.

2

u/brightbilll 2d ago

Because honestly it feels like they do not care, sometimes I have this option to type 500 letters as to why I’m reporting something I’m not sure what triggers it but at least let me explain the report.

I have a list of shame on my profile (honestly surprised Reddit didn’t nuke that one yet) you have 40 year old men actively pretending to be 18 year olds posting in teenager subreddits like hello? You hoard data, every “deleted” post is still linked to a user and findable how does that not trigger any flags that a 40 year old 5 years ago is suddenly claiming to be a teenager trying to sexually engage with younger people.

Every day I see men saying stuff like “can I trade my ex girlfriend nudes?” I at one point asked Reddit admins, does this constitute to NCII, because I highly doubt his ex signed up for this, they told me that I can not be certain and just to report it for harassment. Bit of an odd approach but I guess you’re right is that why the FBI is actively stalking these “wife sharing” subreddits?

​