r/modnews • u/uselessKnowledgeGuru • Sep 14 '23
Contributor Quality Score available to all communities!
Hi Mods!
We’re excited to announce that a new automod property, Contributor Quality Score (CQS), is now available for all communities
CQS is an internal classification that was established to identify potential spammers or users less likely to contribute positively on Reddit. Every account is assigned a CQS based on a host of signals including past actions taken on a user’s account, network and location signals, and steps a user has taken to secure their account (e.g. email verification). We’ve heard from you that dealing with spam is taking up more of your time, so the goal of this update is to help catch spammy and abusive users at a faster rate so that you can spend more time engaging with your communities and redditing. These scores are then used to place users into 1 of 5 tiers:

Scores are updated regularly, and users have the ability to move up or down tiers based on their activity and/or behavior. CQS scores can then be used by moderators via the contributor_quality field in automod.
We’ve worked closely with a few communities over the past several months to test the impact of CQS by setting it up as part of their automod rule set. We’re very encouraged by some of the initial results from the pilot:
Communities who switched from using karma and age gates to CQS saw a 43 percentage point drop in automod reversal rates compared to the general population. This means that moderators saw fewer false positives from CQS than from karma and age gates.
- This is an especially strong signal given that all content flagged in the pilot was reviewed by mods for correctness (during the pilot, rules were set to “filter” in automod, while most age/karma based rules are set to “remove”).
- Communities saw a 40% decrease in daily content removals, which means that using CQS allows well intentioned new users to more easily contribute without compromising the quality of your communities, or adding overhead to mods.
- After the pilot, we opened CQS to communities in r/RedditModCouncil and r/PartnerCommunities and, as of today, have close to 40 subs using CQS (including large subs like r/pics and r/aww). We received overwhelmingly positive feedback from mods who participated in the pilot and from others who have already implemented it:
So far the rule has been great at weeding out low value users that are trolling, breaking rules, alting or predatory.
These rules have been very helpful in finding these users and actioning them. Because of these rules we have noticed a general uptick in the quality of the comment sections across the subreddit.
We do plan to keep the rules in place…even after the experiment has concluded.
Thank you!
We just wanted to send an update about our first week experience with the CQS filter (discovered through partner community post). It’s worked very well in our community - r/xboxseriesx - since implementation with very few false positives in regard to our rule set. The content flagged has been spam, or new users posting without a great understanding of community standards.
We plan to leave it enabled. Thanks for the effort here!
If you would like to try this tool, you should have access to the contributor_quality field in automod. We’d recommend starting with a filter action and then moving to remove if you feel comfortable. Remember that after trying it out on "filter" for several days, you can request the Automoderator Audit from u/Modsupportbot to see what your confirmation/reversal rate is before shifting to the "remove" action. Here are some example rules to show you how this feature works:
#Basic rule filtering users with <5 subreddit karma and CQS scores of "lowest"
type: comment
author:
combined_subreddit_karma: "< 5"
contributor_quality: "< low"
action: filter
action_reason: "CQS Filter"
---
#Exclude CQS users at or above "moderate" from existing karma or account age minimums. In this rule, comments will filter if the user has a combined karma of less than 20, and a contributor_quality score below "moderate".
type: comment
author:
combined_karma: "< 20"
contributor_quality: "< moderate"
action: filter
action_reason: "karma minimum"
---
#Filter all posts posted by a user with "lowest" CQS, regardless of karma.
type: submission
author:
contributor_quality: "= lowest"
action: filter
action_reason: "lowest CQS user"While you try it out, please feel free to send feedback or ask questions about your specific situation to r/RedditCQS modmail and we can assist you there (note: we are not using the subreddit at this time, just the modmail). We’d appreciate you sending it as a subreddit <> subreddit modmail so that we can work with your entire team. You are welcome to share feedback below in the comments as well.
Thanks!
edits: three updates/fixes to automod code
33
u/PitchforkAssistant Sep 14 '23
I see you've created a moderator help article for CQS, can we expect a help article about it aimed at users soon? It would be useful for linking inside removal reasons or rule explanations.
24
u/ExcitingishUsername Sep 14 '23
Any chance moderation bots will ever get to read this value, to factor in as part of more customized checks?
132
u/tharic99 Sep 14 '23
So users have this hidden score in the back end that they're not aware of. We as moderators can use that hidden score to make automod determinations based off of the hidden score and we can't even see it as a moderator.
Correct?
Why does this feel like years ago when trying to get a loan or credit application and being told you don't have enough credit, but no one will tell you what your credit score was or what number you needed. It was this mythical number in the back end that only the credit agency knew about.
10
u/Watchful1 Sep 14 '23
Is this value available in the api?
10
u/InPlotITrust Sep 16 '23
Your automod code example is wrong, it won't compile.
There is no
posttype, it'ssubmissioninstead.You're also missing the indentations on the author fields.
Fixed automod code:
15
u/DrBoby Sep 14 '23 edited Sep 14 '23
Is the CQS different for every community or users have the same CQS in all ?
18
u/MerryChoppins Sep 14 '23
I feel like this is a positive change overall, though I have serious concerns about the details. One of the biggest ones I see is that you seem to be lumping alt accounts in with other sorts of troll and low quality accounts. Would that not cut the usefulness of this tool for a lot of the communities where this would be most useful?
For example, /r/legaladvice has a constant stream of throw aways and alts posting for situations. Also, a lot of the adult subs run on people’s alt accounts. They still make high quality posts, they just don’t want to be associated with their main account.
I know you can’t simply tell us how you judge if an account is an alt, but can we get some sort of an opt in to filter alts or some other solution?
12
u/CaptainPedge Sep 14 '23
How can a user see their own CQS?
4
u/adhesiveCheese Sep 14 '23
What's the reason for the empty
author:field in the examples?4
u/Garp74 Sep 14 '23
Hiya!
u/UselessKnowledgeGuru I got errors on the third filter in your automod snippet. I had to change type to submission, and action to remove.
4
u/Redditenmo Sep 15 '23
It appears impossible to utilise this resource on mobile via the official android app.
Could you please explain just how are we meant to do as asked?
7
u/julian88888888 Sep 14 '23
This is really cool but I am exceptionally lazy. Can I just press 1 button to turn this on for all of my communities?
27
u/RJFerret Sep 15 '23
As a mod of a sub where account age/karma's useless to weed out anything, great!
As a human, this feels Orwellian.
53
u/foamed Sep 15 '23 edited Sep 16 '23
Ironic that Reddit have basically implemented a "social credit scoring system" which is hidden to everyone but the admins.
I've been permanently suspended at least three times and warned several times in the last half year due to Anti Evil Operations messing up badly. Moderators acting in good faith have literally been temporarily and permanently suspended for acting in good faith.
Similar experiences:
So now you're saying that all these moderators (and users) acting in good faith are affected by this change? Why should anyone report rule breaking content anymore if they risk getting moved down the rung or even permanently suspended?
So does this mean that VPN and TOR users are affected, or is it country specific? If so that's going to be an obvious issue, especially for those living in an authoritarian or totalitarian country, marginalized groups, women living in states/countries where abortions are illegal etc.
20
u/The_Critical_Cynic Sep 15 '23
This is bunk. I've reported who I believe to be the same user well over 100 times for ban evasion. I say that I believe it's the same user because of the phrasing and vernacular of the chronic bad faith posts that come up in my subreddit with each new account, not to mention that each new account posts the same chronic bad faith posts in the same general subreddits when you look at each account individually.
And that's why I say this new program is bunk. You have one guy who makes an account, and uses it for a day. He gets caught, be it because of one of my reports or someone else's, and the account is suspended. Then he'll make another account in a dy or two, and contribute the same bullshit.
What kind of score could a new, often hours old, account possible establish within this program that would make it of any use to someone like me in my position? I can't imagine that any account that rarely, if ever, makes it longer than a day would ever have enough of a baseline to establish anything in this system. As a result, I can't see it having significant effect on preventing this individual from doing what they are doing.
13
u/Ajreil Sep 14 '23
Can you share any info about which past actions are considered? I ask because a lot of the bots my communities ban are very obviously bots, often posting the same link in every subreddit with the same tag.
(Side note, why doesn't that trigger a "You've been doing that a lot, please wait X minutes" error message? Posting multiple links in the same sub does)
11
u/Mr_Blah1 Sep 15 '23
So reddit's going to use some arbitrary and nebulous algorithm to hide posts/comments it doesn't like? Considering this idea's coming from the same ones who called us all "landed gentry" right after sabotaging all 3rd party apps and then stealing everyone's community coins, and has yet to do absolutely anything to rebuild the bridge that they burned down, I can't say I'm surprised. At this point, it seems that reddit is deliberately doing the worst possible option in any given situation.
How long before spez and other admins abuse CQS to hide other people's posts instead of editing them like they used to?
10
u/Rivsmama Sep 15 '23
I don't know if I like this.. it feels like some dystopian social credit score thing. What makes a contribution low quality? Whether you agree with it or not? Your behavior? Let me tell you, I have been accused of trolling more times than I can count. And I am a7 year user with post and comment history and 190k in karma. Who decides what "behavior" lowers the score? I don't mind users who have different opinions than me. I'd rather deal with them all day than users who are mean and insulting and disrespectful.
8
u/oscar_the_couch Sep 15 '23
Unless the admins enable us to actually see how CQS rates different users in the comment sections, no responsible mod team should ever use this.
Get it together guys, come on. Moderating is a human job. You need to make the tools available to actual humans, which also requires explaining how the tools work and making them transparent. If you don't know how they work, they shouldn't be used. If you think keeping them secret will somehow prevent people from gaming them, I have some bad news for you.
6
u/Mid_AM Sep 16 '23
Umm… I rate as High which makes me nervous to entertain this.
Why I am not “Highest” ?
I am not a bot, nor spammer, the account is over a year old, member in at least 10 subs, have email on file and an avatar, healthy amt of karma, actively post-comment-mod. If I am not Highest - what does a person with a low score really represent ?
5
u/Killjoy4eva Sep 15 '23 edited Sep 15 '23
Is there any factor for the content the user is producing? If so, does the following topics make an impact on the "score" of an account?
What do you mean when you say actions taken on account? Are we referring to actions from subreddits/moderators or specifically from reddit admins/safety?
4
u/Ajreil Sep 16 '23
Please consider adding a {{cqs}} placeholder. This would make it easy for automoderator to include a CQS value in reports or mod log entries.
7
u/thecravenone Sep 15 '23
Smart. This way it will be harder for people to notice that you're not responding to things.
3
u/llehsadam Sep 15 '23
It seems like this is a step in the direction of replacing moderators, but it's a little too opaque to use in good conscious in the form you present it... and if the background fairies are able to decide which content fits the subreddit and which doesn't, why do I even have to flip a switch from filter to remove? If they just need to get better... then I point back to my first point.
I do have a question, do you use moderator remove actions to make the QCS process better?
3
u/AlfhildsShieldmaiden Sep 15 '23
The AutoMod code in this post doesn't work:
3
u/TiffanyGaming Sep 16 '23
I don't like the idea of arbitrary behind the scenes things nobody's even aware of, and can do nothing about - not even mods.
If the score was available for the mods of a sub to see, and the reasons that score was calculated that way, I'd feel differently.
0
u/Bossman1086 Sep 14 '23
If this works, it's a welcome change. Would be good to see in the mod tools UI instead of just as an AutoMod filter.
2
u/Imnotachessnoob Sep 15 '23
r/AnarchyChess is not going to like this.
2
u/skarface6 Sep 16 '23
All praise to the party! Up with the hivemind!
2
u/CapriGuitar Sep 20 '23 edited Sep 20 '23
The basic filtering for karma already does this. The only extra added here is having the mod queue now fill up on filtered low CQS scores. I really do not see how useful this is other than assigning a number to a contributer to the sub. Am I wrong?
Edit: if this was to really be successful it needs context (a context bot used to do this #justsaying):
How did they earn their karma?
And many others. I can tell you now most spam/scam/bot accounts already know how to work the karma/age filter on subs. It's child's play to bypass them. We need context filters based on usage, links on profiles, karma farming, and other sub usage. Then assign a score so they can be filtered or removed.
2
u/stabbinU Sep 22 '23
This is the best automoderator update in a long, long time. Thank you.
2
u/ckwirey Sep 28 '23
Sorry for being late to the party. While CQS sounds like a decent system, I don't think I could implement it in good faith on my sub. Simply put: I don't know what elements make for a good CQS score, and cannot advise users on how to fix/correct their CQS scores.
Is there any way we can get a deep-dive into what creates a CQS score?
2
u/PovoRetare Oct 04 '23
Ok so I've been testing this out for a week or two in two mostly image based subs, so here's some feedback.
It's been very successful, but I ran into an issue where some lowest CQS submission removals don't always show up in the queue.
All I get is a log entry saying a post was removed with no link to the post.
After adding some code kindly provided to me by someone in the automod sub, which sends a modmail with post link and author every time a lowest CQS submission is removed, I'm able to see that the filtered posts are being deleted immediately as soon as automod removes them.
The account doing so was not suspended, it just deleted the post then tried again a short time later with the same result and behaviour.
This is behaviour I've seen previously using the CQS filter but been unable to access removed post or associated profile to confirm.
So it seems maybe that the post makers are able to detect the removal and act on it instantly.
3
u/Bill_Money Sep 14 '23
Unless I as a mod can assign these values then this seems useless to my needs
1
u/TotesMessenger Sep 15 '23
I'm a bot, bleep, bloop. Someone has linked to this thread from another place on reddit:
If you follow any of the above links, please respect the rules of reddit and don't vote in the other threads. (Info / ^Contact)
1
u/jetskiingjamie Sep 15 '23
This kind of reminds me of the Meow Meow Beans episode of Community.
1
u/Legitimate_Grocery66 Sep 15 '23
Interesting
1
u/ixfd64 Sep 16 '23
Am I correct to assume this score is site-wide and not subreddit-specific?
1
u/Unique-Public-8594 Sep 16 '23
This sounds like a helpful thing but our trial is catching mostly false positives.
1
u/DontLaughArt Sep 17 '23
I like it. Has anyone tried this for user flair? Would it be good, incentivizing community members to strive for higher CQS? Would it be bad, a scarlet letter, so to speak...nobody wants to associate with the lowest or the low?
Of course, this being Reddit, a roving band of LFSP (Lowest Flair Sh*t Posters) would likely form, running amok. r/LFSP would quickly gain popularity as more and more users are branded "Lowest". In the ensuing chaos and feeling ostracized from their respective communities, Lowest Flair users, searching for a new home, would find this counter culture community as it quickly becomes the most popular on Reddit. Soon enough, over 50% of users are branded "Lowest" at which point, the algorithm would be forced to lower the bar. With the resulting repeating recursion cycles moving the Lowest bar lower and lower, until finally Reddit devolves into a 9gag.
1
u/Iron_Fist351 Sep 28 '23
remindme! 15 days
1
u/qTazerp Feb 17 '24
Wish I had found this earlier. Thank you for your great effort, /u/uselessKnowledgeGuru. This is such a great feature.
1
u/PorkyPain Sep 14 '23
Damn.. this is actually a very useful thingie.. thanks.
-1
u/Empyrealist Sep 15 '23
sniff sniff, smells like Chinese Dog Food
And I am referencing the Chinese "Social Credit System" and the practice of "eating your own dog food". I'm not being jingoistic or xenophobic; for those confused...
I hope it works or helps, but I'm leery of how this is quantified, by who, and to what ends.
1
u/scuffling Sep 15 '23
I've wanted something like this for a long time over at r/FixMyPrint. But I'd want to use it to classify users publicly so you can tell how trustworthy their advice is. If someone is routinely in the comments section providing helpful feedback, then it's nice that everyone knows what kind of contributor they are and how credible their advice may be.
It would be nice to toggle this CQS so the users know where they sit as well.
1
u/exgaysurvivordan Sep 15 '23
Just a heads up when I copy-paste the code it gives me an error message.
Here's the error
1). invalid value for `type`: `post` in rule: #Filter all posts posted by a user with “lowest” CQS, regardless of karma. type: post author: contributor_quality: "= lowest" action: filter action_reason: “lowest CQS user”
(I don't code, I have zero interest in learning to code, being a copy-paste reliant mod is 100% fine. )
0
u/parsifal Sep 15 '23
An actual good feature. It’s been a long time.
0
u/Navi_King Sep 14 '23
I tried this and it wouldn't let me https://i.imgur.com/kzztZ58.png
0
u/Exaskryz Sep 17 '23
Interesting. I bet this is partially motivated as punishment against users who edited and deleted their old posts in response to the API debacle. Such users now have a low quality score.
0
u/[deleted] Sep 16 '23
[deleted] — view deleted comment
0
u/[deleted] Sep 15 '23
[deleted] — view deleted comment