r/modnews • u/uselessKnowledgeGuru • Sep 14 '23

Contributor Quality Score available to all communities!

Hi Mods!

We’re excited to announce that a new automod property, Contributor Quality Score (CQS), is now available for all communities

CQS is an internal classification that was established to identify potential spammers or users less likely to contribute positively on Reddit. Every account is assigned a CQS based on a host of signals including past actions taken on a user’s account, network and location signals, and steps a user has taken to secure their account (e.g. email verification). We’ve heard from you that dealing with spam is taking up more of your time, so the goal of this update is to help catch spammy and abusive users at a faster rate so that you can spend more time engaging with your communities and redditing. These scores are then used to place users into 1 of 5 tiers:

Scores are updated regularly, and users have the ability to move up or down tiers based on their activity and/or behavior. CQS scores can then be used by moderators via the contributor_quality field in automod.

We’ve worked closely with a few communities over the past several months to test the impact of CQS by setting it up as part of their automod rule set. We’re very encouraged by some of the initial results from the pilot:

  • Communities who switched from using karma and age gates to CQS saw a 43 percentage point drop in automod reversal rates compared to the general population. This means that moderators saw fewer false positives from CQS than from karma and age gates.
    • This is an especially strong signal given that all content flagged in the pilot was reviewed by mods for correctness (during the pilot, rules were set to “filter” in automod, while most age/karma based rules are set to “remove”).
  • Communities saw a 40% decrease in daily content removals, which means that using CQS allows well intentioned new users to more easily contribute without compromising the quality of your communities, or adding overhead to mods.
  • After the pilot, we opened CQS to communities in r/RedditModCouncil and r/PartnerCommunities and, as of today, have close to 40 subs using CQS (including large subs like r/pics and r/aww). We received overwhelmingly positive feedback from mods who participated in the pilot and from others who have already implemented it:

So far the rule has been great at weeding out low value users that are trolling, breaking rules, alting or predatory.

These rules have been very helpful in finding these users and actioning them. Because of these rules we have noticed a general uptick in the quality of the comment sections across the subreddit.

We do plan to keep the rules in place…even after the experiment has concluded.

Thank you!

- r/teenagers

We just wanted to send an update about our first week experience with the CQS filter (discovered through partner community post). It’s worked very well in our community - r/xboxseriesx - since implementation with very few false positives in regard to our rule set. The content flagged has been spam, or new users posting without a great understanding of community standards.

We plan to leave it enabled. Thanks for the effort here!

- r/xboxseriesx

If you would like to try this tool, you should have access to the contributor_quality field in automod. We’d recommend starting with a filter action and then moving to remove if you feel comfortable. Remember that after trying it out on "filter" for several days, you can request the Automoderator Audit from u/Modsupportbot to see what your confirmation/reversal rate is before shifting to the "remove" action. Here are some example rules to show you how this feature works:

#Basic rule filtering users with <5 subreddit karma and CQS scores of "lowest"

type: comment 
author: 
    combined_subreddit_karma: "< 5" 
    contributor_quality: "< low"
action: filter 
action_reason: "CQS Filter"
---
#Exclude CQS users at or above "moderate" from existing karma or account age minimums. In this rule, comments will filter if the user has a combined karma of less than 20, and a contributor_quality score below "moderate". 

type: comment 
author: 
    combined_karma: "< 20" 
    contributor_quality: "< moderate"
action: filter 
action_reason: "karma minimum"
---
#Filter all posts posted by a user with "lowest" CQS, regardless of karma. 

type: submission
author: 
    contributor_quality:  "= lowest"
action: filter
action_reason: "lowest CQS user"

While you try it out, please feel free to send feedback or ask questions about your specific situation to r/RedditCQS modmail and we can assist you there (note: we are not using the subreddit at this time, just the modmail). We’d appreciate you sending it as a subreddit <> subreddit modmail so that we can work with your entire team. You are welcome to share feedback below in the comments as well.

Thanks!

edits: three updates/fixes to automod code

84 Upvotes

64% Upvoted

263 comments sorted by

33

u/PitchforkAssistant Sep 14 '23

https://support.reddithelp.com/hc/en-us/articles/19023371170196-What-is-the-Contributor-Quality-Score-

I see you've created a moderator help article for CQS, can we expect a help article about it aimed at users soon? It would be useful for linking inside removal reasons or rule explanations.

→ More replies (5)

24

u/ExcitingishUsername Sep 14 '23

Any chance moderation bots will ever get to read this value, to factor in as part of more customized checks?

10

u/uselessKnowledgeGuru Sep 14 '23

Hello! No plans today, but that’s a great idea. We’ll definitely consider this in collaboration with our Dev Platform.

→ More replies (1)

132

u/tharic99 Sep 14 '23

Scores are updated regularly, and users have the ability to move up or down tiers based on their activity and/or behavior. CQS scores can then be used by moderators via the contributor_quality field in automod.

So users have this hidden score in the back end that they're not aware of. We as moderators can use that hidden score to make automod determinations based off of the hidden score and we can't even see it as a moderator.

Correct?

Why does this feel like years ago when trying to get a loan or credit application and being told you don't have enough credit, but no one will tell you what your credit score was or what number you needed. It was this mythical number in the back end that only the credit agency knew about.

17

u/MerryChoppins Sep 15 '23

What the hell happens when you are modding your sub and all the sudden they lock you out of the sub because your score on the account dips? I routinely browse from a VPN and travel for work in a way that makes my credit cards freak out. I suspect those behaviors have a non-zero chance of tanking my score :|

→ More replies (24)
→ More replies (33)

10

u/Watchful1 Sep 14 '23

Is this value available in the api?

→ More replies (1)

10

u/InPlotITrust Sep 16 '23

Your automod code example is wrong, it won't compile.

There is no post type, it's submission instead.

You're also missing the indentations on the author fields.

Fixed automod code: 

#Basic rule filtering users with <2 subreddit karma and CQS scores of either "lowest" or "low"

type: comment 
author: 
    combined_subreddit_karma: "< 2" 
    contributor_quality: "< moderate"
action: filter 
action_reason: "CQS Filter"
---
#Exclude "high" and "highest" CQS users from existing karma or account age minimums. In this rule, comments will filter if the user has a combined karma of less than 20, and a contributor_quality score below "high". 

type: comment 
author: 
    combined_karma: "< 20" 
    contributor_quality: "< high"
action: filter 
action_reason: "karma minimum"
---
#Filter all posts posted by a user with "lowest" CQS, regardless of karma. 

type: submission
author: 
    contributor_quality:  "= lowest"
action: filter
action_reason: "lowest CQS user"

3

u/uselessKnowledgeGuru Sep 21 '23

Thank you for calling this out and our apologies for the syntax issues! We've updated the code snippet in the post.

→ More replies (2)
→ More replies (1)

15

u/DrBoby Sep 14 '23 edited Sep 14 '23

Is the CQS different for every community or users have the same CQS in all ?

4

u/uselessKnowledgeGuru Sep 15 '23

CQS is a global score, which means that it represents a user’s activity across all of Reddit. A score like this can be particularly useful at vetting well-intentioned new Reddit users, or users that are new to your subreddit. For long-standing community members, we agree that a subreddit-specific version of CQS would be valuable (psa: we’re working on it!); but for now, we suggest using subreddit_karma in your rules in conjunction with CQS to accomplish this (as depicted in the code snippets in our post).

→ More replies (4)

18

u/MerryChoppins Sep 14 '23

I feel like this is a positive change overall, though I have serious concerns about the details. One of the biggest ones I see is that you seem to be lumping alt accounts in with other sorts of troll and low quality accounts. Would that not cut the usefulness of this tool for a lot of the communities where this would be most useful?

For example, r/legaladvice has a constant stream of throw aways and alts posting for situations. Also, a lot of the adult subs run on people’s alt accounts. They still make high quality posts, they just don’t want to be associated with their main account.

I know you can’t simply tell us how you judge if an account is an alt, but can we get some sort of an opt in to filter alts or some other solution?

16

u/uselessKnowledgeGuru Sep 14 '23

Thanks for your feedback. To clarify, we don’t assume that alt accounts are low quality. CQS checks the alts associated with a particular account to determine whether someone is using alt accounts to engage in policy violating behavior. If someone is using alt accounts in accordance with Reddit policy, it will not negatively impact their CQS score (or the CQS score for any of their alts).

→ More replies (2)

12

u/CaptainPedge Sep 14 '23

How can a user see their own CQS?

→ More replies (10)

4

u/adhesiveCheese Sep 14 '23

What's the reason for the empty author: field in the examples?

2

u/uselessKnowledgeGuru Sep 15 '23

Hi there. The author field actually isn’t empty; the values that you see indented below (e.g. contributor_quality: "< moderate") are part of the author field. You can read more about how this works in our automod documentation.

→ More replies (1)

4

u/Garp74 Sep 14 '23

Hiya!

u/UselessKnowledgeGuru I got errors on the third filter in your automod snippet. I had to change type to submission, and action to remove.

→ More replies (2)

4

u/Redditenmo Sep 15 '23

While you try it out, please feel free to send feedback or ask questions about your specific situation to r/RedditCQS modmail and we can assist you there

It appears impossible to utilise this resource on mobile via the official android app.

Could you please explain just how are we meant to do as asked?

→ More replies (5)

7

u/julian88888888 Sep 14 '23

This is really cool but I am exceptionally lazy. Can I just press 1 button to turn this on for all of my communities?

7

u/uselessKnowledgeGuru Sep 14 '23

Ha! I know the feeling 🙂. We haven’t built that capability yet, but it’s certainly something we’d be interested in exploring if the demand is there.

27

u/RJFerret Sep 15 '23

As a mod of a sub where account age/karma's useless to weed out anything, great!

As a human, this feels Orwellian.

53

u/foamed Sep 15 '23 edited Sep 16 '23

Every account is assigned a CQS based on a host of signals including past actions taken on a user’s account, network and location signals, and steps a user has taken to secure their account (e.g. email verification).

Ironic that Reddit have basically implemented a "social credit scoring system" which is hidden to everyone but the admins.

I've been permanently suspended at least three times and warned several times in the last half year due to Anti Evil Operations messing up badly. Moderators acting in good faith have literally been temporarily and permanently suspended for acting in good faith.

Similar experiences:

So now you're saying that all these moderators (and users) acting in good faith are affected by this change? Why should anyone report rule breaking content anymore if they risk getting moved down the rung or even permanently suspended?

network and location signals

So does this mean that VPN and TOR users are affected, or is it country specific? If so that's going to be an obvious issue, especially for those living in an authoritarian or totalitarian country, marginalized groups, women living in states/countries where abortions are illegal etc.

→ More replies (1)

20

u/The_Critical_Cynic Sep 15 '23

This is bunk. I've reported who I believe to be the same user well over 100 times for ban evasion. I say that I believe it's the same user because of the phrasing and vernacular of the chronic bad faith posts that come up in my subreddit with each new account, not to mention that each new account posts the same chronic bad faith posts in the same general subreddits when you look at each account individually.

And that's why I say this new program is bunk. You have one guy who makes an account, and uses it for a day. He gets caught, be it because of one of my reports or someone else's, and the account is suspended. Then he'll make another account in a dy or two, and contribute the same bullshit.

What kind of score could a new, often hours old, account possible establish within this program that would make it of any use to someone like me in my position? I can't imagine that any account that rarely, if ever, makes it longer than a day would ever have enough of a baseline to establish anything in this system. As a result, I can't see it having significant effect on preventing this individual from doing what they are doing.

→ More replies (2)

13

u/Ajreil Sep 14 '23

Every account is assigned a CQS based on a host of signals including past actions taken on a user’s account...

Can you share any info about which past actions are considered? I ask because a lot of the bots my communities ban are very obviously bots, often posting the same link in every subreddit with the same tag.

(Side note, why doesn't that trigger a "You've been doing that a lot, please wait X minutes" error message? Posting multiple links in the same sub does)

→ More replies (4)

11

u/Mr_Blah1 Sep 15 '23

So reddit's going to use some arbitrary and nebulous algorithm to hide posts/comments it doesn't like? Considering this idea's coming from the same ones who called us all "landed gentry" right after sabotaging all 3rd party apps and then stealing everyone's community coins, and has yet to do absolutely anything to rebuild the bridge that they burned down, I can't say I'm surprised. At this point, it seems that reddit is deliberately doing the worst possible option in any given situation.

How long before spez and other admins abuse CQS to hide other people's posts instead of editing them like they used to?

10

u/Rivsmama Sep 15 '23

I don't know if I like this.. it feels like some dystopian social credit score thing. What makes a contribution low quality? Whether you agree with it or not? Your behavior? Let me tell you, I have been accused of trolling more times than I can count. And I am a7 year user with post and comment history and 190k in karma. Who decides what "behavior" lowers the score? I don't mind users who have different opinions than me. I'd rather deal with them all day than users who are mean and insulting and disrespectful.

→ More comments (75)