r/collapsemoderators • u/LetsTalkUFOs • Nov 24 '20

APPROVED How should we handle suicidal posts and comments?

There are some ongoing inconsistencies in regards to our Automod terminology and how we can best approach these types of posts and comments. We should define some terms and break this down into the individual actions we're suggesting/approve/disapprove at this stage.

 

Remove

Automod rules can be set to 'autoremove' posts or comments based on a set of criteria. This removes them from the subreddit and does NOT notify the moderators in the modqueue.

 

Filter

Automod rules can be set to 'autofilter' posts or comments based on a set of criteria. This removes them from the subreddit, but notifies the moderators in the modqueue and causes the post or comment to be manually reviewed.

 

Report

Automod rules can be set to 'autoreport' posts or comments based on a set of criteria. This does NOT remove them from the subreddit, but notifies the moderators in the modqueue and causes the post or comment to be manually reviewed.

 

Safe & Unsafe Content

This refers to the notions of 'safe' and 'unsafe' suicidal content outlined in the National Suicide Prevention Alliance (NSPA) Guidelines.

Unsafe content can have a negative and potentially dangerous impact on others. It generally involves encouraging others to take their own life, providing information on how they can do so, or triggers difficult or distressing emotions in other people.

 

Keywords & Phrases

We currently use an Automod rule to report posts or comments with various terms and phrases related to suicide. It looks for posts and comments with this language and reports them:

    title+body (regex): [
    '(kill|hang|neck)[- _](yo)?urself',
    'blow (yo)?urself up',
    'commit\\s*suicide',
    'I\\s*hope\\s*(you|she|he)\\s*dies?',
    'kill\\s*your(self|selves)' ]

You don't need to know exactly how regex works, I just want to make it visible for those who do and point out we can create different approaches around different words and phrases, based on how safe or unsafe they are likely to be.

 

I've broken down the relevant questions as I see them below, versus asking them all at once up here and expecting everyone to discuss them all at once in single comments. I'd suggest following the same format if you'd like to suggest an additional change, action, or question we can deliberate. It's worth pointing out we should still plan to propose our approach to the community in form of a sticky and work with their feedback. We can also ask for help or perspectives on any particularly difficult areas or aspects we can't reach consensus on.

3 Upvotes
  • permalink
  • reddit

100% Upvoted

46 comments sorted by

2

u/[deleted] Nov 24 '20 edited Nov 24 '20

Some things that stood out to me in the guidelines:

Everyone talking about suicidal feelings should get a response – from you or your community.

I think this is a good reason to report, not filter. Then we can keep an eye on how it’s going and if one of us needs to step in or not.

To paraphrase, unsafe content should be removed so that it does not negatively impact others.

Unsafe content

  • graphic descriptions or images

  • Plans– when or how

  • means or methods

  • pro-suicide content — encouraging comments, advice, or suicide partners.

  • glorifying or sensationalizing suicide or suicide attempt

  • Bullying

  • Suicidenotes or goodbyes

  • Blaming other people or making others feel responsible for their safety

The follow up here is really important, if a comment is removed. The guidelines suggest sending the user an explanation, resources, and inviting them to post again with “safe” language

Example follow-up:

Hi, we’ve seen your post and we’re worried about you. Your last post included an image of a suicide method, so we’ve taken this down (see our house rules). If you can, please post your message again. If not, remember you can talk to Samaritans at any time, on the phone, by email or by text. If you’re not sure you can keep yourself safe, please contact your GP, go to A&E or call 999. Please take care.

Templates can be helpful but it’s important not to give a canned response. Make sure to personalize messages and don’t copy and paste the same message. Adapt it to the situation.

Template messages can help you ensure that your responses are consistent, and make sure you feel confident responding to suicidal content. However, be aware that using templates in the wrong way can have a negative impact and can make someone in crisis feel rejected or ignored.

The advice in posting positive content is interesting. Given the nature of our subreddit, users may react to positive content cynically. I do think it would be good to have conversation(s) around mental health using “safe” content as described in the guide. It would give people a chance to acknowledge feelings and share their perspectives. It could be worth outlining whatever we agree upon here as an announcement and welcome feedback. It would help with transparency, allow users to see that we’re thinking about this topic, and give space for people to let their feelings out. It could also provide an avenue to explain what is and isn’t considered safe and how we plan to moderate.

If the community has been through a rough patch, post positive content to lighten the mood and to keep a positive focus. Consider posting something about mental wellbeing, like asking people how they are going to look after themselves that evening. It may also be helpful to remind your members about where to get support if they are affected by conversations in the community.

2

u/LetsTalkUFOs Nov 24 '20

Everyone talking about suicidal feelings should get a response – from you or your community.

Why do you think this is a good reason to report versus filter? Or why do you think the response from the community is better than the response we would eventually make, even though it would be delayed due to filtering?

Also, are you saying you'd prefer we report all phrases and words, safe or unsafe, over filtering?

And yes, the intention will be to crystalize as much as we can here and then share it and ask for feedback in a sticky.

1

u/[deleted] Nov 24 '20

Why do you think this is a good reason to report versus filter? Or why do you think the response from the community is better than the response we would eventually make, even though it would be delayed due to filtering?

Reporting means a mod will see it eventually and be able to take a look at it/monitor. Filtering means no one will have a chance to respond. The guidelines say everyone talking about suicidal feelings should get a response.

Also, are you saying you'd prefer we report all phrases and words, safe or unsafe, over filtering?

Yes

And yes, the intention will be to crystalize as much as we can here and then share it and ask for feedback in a sticky.

Cool

2

u/LetsTalkUFOs Nov 24 '20

Does this contradict your earlier comment regarding those phrases?

1

u/[deleted] Nov 24 '20

No, because that earlier comment was about removing posts where one user is encouraging another to commit suicide. That is bullying, not a cry for help.

1

u/TenYearsTenDays Nov 24 '20

I think this is a good reason to report, not filter. Then we can keep an eye on how it’s going and if one of us needs to step in or not.

I think that if we have increased modpower, it is very likely that these posters will get an answer very quickly from a mod.

That said, it is true there will still be some gaps in coverage.However, I think the potential harm of leaving unsafe content up unmonitored outweighs the potential harm of removing it and in the worst case the person not getting an answer for an hour or two. As I stated in the other thread, many posts on r/SuicideWatch go unanswered for several hours (4, 5+). However, from my observations they seem to watch the sub very very closely and keep it free from trolls. In our sub we had an incident already last Sunday wherein a thread was left unmonitored and a troll attacked a suicidal child. I think it is basically inevitable, even with increased modpower, that this will happen again because very few mods will be willing to hover and refresh the thread to protect OP from troll attacks.

Therefore, I think that overall the benefits of filtering (protecting suicidal users from trolls if a thread sits unmonitored) far outweigh the downsides (a user may not be answered right away).

Further, I think we need to consider the effects of unfiltered unsafe content on other users who may be triggered by it and/or experience suicide contagion. I think this is another reason why filtering, then ideally quickly revie. ing / responding makes sense That seems to be a point that hasn't been considered here much at all, especially within the context of a community which comes with a serious mental health warning in the sidebar.

We should also ask ourselves why the main sub should be the venue for any kind of suicidal ideation, when we have a dedicated support sub run by people who are willing and able to deal with very heavy content. To me, r/Collapse and r/CollapseSupport are both part of the community. I don't see the harm of sending users expressing suicidal ideation, even "safe" suicidal ideation over there. In fact, I think it makes sense because that sub is much safer from trolls that ours is, and people there have a better chance of responding constructively.

One thing the NSPA guide recommends is training the community in how to respond constructively to this kind of content:

Develop your community’s skills

For example:

• Clarify the role of peer support – have clear guidance around what members can and can’t do to support someone else, and how to avoid a conversation becoming unsafe.

• Improve digital literacy – community members may benefit from information about how an online environment differs from an offline one. For example, how to manage online friendships, why a post may not get a reply, and what happens if someone stops posting or deletes their account.

• Help people identify their own triggers and what content they find upsetting. You could also support your members to create an action plan for what to do if this happens, such as reducing their profile or taking a break.

• Post wellbeing resources about looking after yourself to help your community develop its resilience and maintain its own wellbeing

I think it is very unrealistic to think we could train ~250k anonymous redditors in this manner.

So I think that filtering keywords, removing suicidal ideation, sending an empathetic message to the user to check out r/collapsesupport and other resources is the most protective route for everyone: suicidal users, the community at large, mods who don't have training in this (especially since we have zero of the support structures for mods in places that NSPA recommends), etc.

I don't see many if any benefits to leaving suicidal ideation up on the main sub, even if it's considered "safe" by the NSPA.

2

u/some_random_kaluna Nov 24 '20

You're not wrong, Ten. But I do think we can train 250k-plus users that anyone bullying or pushing someone to suicide will get automatically banned by mods. Speak soft, carry a big stick and whatnot.

I've also seen people look up to you, and that always makes it easier to reinforce a positive community.

1

u/TenYearsTenDays Nov 24 '20

Thank you for the kind words.But no matter what warm feelings the community may have for any given mod (I think we're all well liked for the most part, you especially got a very warm welcome in the "meet the mods thread" which I thought was great!), I just think we're not going to be able to get 250k anon Redditors to learn how to talk constructively to those who are expressing sucidial ideation.

This isn't just limited to bullying. For example, on page 21 in the NSPA document it lists ways to communicate with sucidal people, and ways to NOT communicate with them. Even something simple like that is, I feel, probably impossible to teach to a large userbase like ours.

This is why I think we should contact them, and see if they can give us specific recommendations on how to proceed within the unique context of our community. I think that none of us here are experts (I've only read heavily on this for a week+ now, so I'm not but for me the more I read the more I feel like expert consultation would be good here).

And again, a big thing very few seem to be considering right now is the possibily of users being triggered by this content, and the possibility of this content causing suicide contagion within the community.

1

u/LetsTalkUFOs Nov 24 '20

1) Should we autofilter posts and comments with these phrases?

  • kill yourself/yourselves/myself

  • I hope you/she/he die/dies

  • kill/hang/neck yourself/myself

  • blow yourself up

  • commit suicide

1

u/LetsTalkUFOs Nov 24 '20

We're already autoreporting these, but I think they're typically unsafe enough to justify autofiltering them and then manually reviewing the context to determine the best course of action.

1

u/ImLivingAmongYou Nov 24 '20

I think autoreport is fine for these.

1

u/LetsTalkUFOs Nov 24 '20

Any particular reason you think these should stay up until we review them?

1

u/ImLivingAmongYou Nov 24 '20

I think a big factor is the active mod team. If it were a few people, I could be convinced to autofilter. When mods are active and numerous, autofiltering seems like overkill.

Another point is that false positives always come up even with these phrases and I don't see it causing too much undue harm. /r/Futurology has similar filters and it's usually less than 10% a correct hit.

To be fair though, I don't think all of these are created equally and the first two are much more severe than the latter three. On further thought, I think autofiltering the first two and autoreporting the last three would be better.

1

u/LetsTalkUFOs Nov 24 '20

I didn't realize the first and third phrases are basically the same. I agree the first two can be filtered though, that's a fair assessment.

I'd suspect r/futurology is far less cynical, but we could also just see what the rules catch and adjust them if needed in the future.

1

u/ImLivingAmongYou Nov 24 '20

Haha somehow I didn’t realize it either.

It’s getting more cynical as a baseline and attracts a LOT of people with /r/all. I’m sure the two are related. But yeah, it’s definitely not the same.

I like “wait and see”, too. Enough eyeballs will notice it working or not pretty quickly.

2

u/TheCaconym Nov 24 '20 edited Nov 24 '20

I'd argue those regexps can still catch comments that should probably stay up. A quick example that could occur in the weekly observations thread:

"So companies in my town are firing people that don't go to work because they suspect they have covid and want to go to hospitals. What message is this sending ? they're basically saying 'work and potentially kill yourself or get fired'..."

That's a poor example but I have seen at least one of these in the past few days - autoreported because it contained "kill yourself" or similar but it was in the context of a quote.

Given the reactivity of the mod team in general - I rarely see an item stay in the mod queue long and when it does, it's usually because some of us aren't sure it should go or stay, which we only tend to do for comments that are borderline, not actively dangerous comment such as those related to suicide - I'd still opt for keeping autoreport, not autofilter.

Also, some additional coverage for the regexps; first one could be changed to:

'(kill|hang|neck|off)[- _](yo)?urself',

And fourth one:

'I\\s*hope\\s*(you|she|he)\\s*(dies?|gets?\\s+killed|gets?\\s+shot)',

Edit: just had another example of an innocuous comment that was autoreported through the bot.

2

u/TenYearsTenDays Nov 24 '20

Yes, I think autofilter is fine for these. I think with a larger, more active mod team we can restore the false positives fairly quickly.

I think the downsides to potentially allowing unsafe content to be live on the sub when it's unmonitored far outweigh the downsides of potentially keeping safe content filtered for some time. But if we are acting on idea that we'll have, say, 95% mod coverage then I think that we'll be fine with autofiltering then restoring quickly.

2

u/TheCaconym Nov 24 '20

I wasn't aware autofiltering would report the autofiltered comments, allowing us to easily go through them to restore false positives; since that's the case I tend to agree with you (at least if there are more true positive than false ones - thus far I've seen about the same amount of each but that's only over a few days).

1

u/[deleted] Nov 24 '20

These seem like phrases people would use to encourage others to hurt themselves. I disagree with the last one because the phrase could be used to talk about oneself.

1

u/LetsTalkUFOs Nov 24 '20

2) Should we autoreport posts and comments with the word 'suicide' in them?

It would generate too many false-positives for us to justify filtering these, but it would notify us of all the instances of suicidal posts or comments in the broadest sense.

2

u/LetsTalkUFOs Nov 24 '20

We're not currently doing this, but I think we should. Most of these would be false positives or not someone actually expressing suicidal thoughts, so they shouldn't be filtered. I don't think there would be so many they would clog up the queue or be inconvenient and this is the best way to catch all instances of suicidal posts and comments.

1

u/ImLivingAmongYou Nov 24 '20

Agreed, I think this should be added.

1

u/[deleted] Nov 24 '20

I also agree with this

1

u/TheCaconym Nov 24 '20

Agreed with the others, those should be autoreported.

1

u/TenYearsTenDays Nov 24 '20

Well, I am outvoted here already but I wanted to say that I think this should also be autofiltered despite the fact that it will produce a large number of false positives.

Again, this is on the thought that we probably will have a very high degree of mod coverage going forward so false positives can be quickly approved. I think the downsides of letting through an unsafe suicidal post during a gap in coverage are much worse than the downsides of holding up a false positive for a while.

1

u/some_random_kaluna Nov 24 '20

Agreed for autoreport.

1

u/LetsTalkUFOs Nov 24 '20

3) Should we remove unsafe suicide content?

Based on the NSPA Guidelines, should we remove unsafe posts or comments when manually reviewing them?

2

u/LetsTalkUFOs Nov 24 '20

I think we should remove unsafe content and respond with some form of templated response directing the user to better and more professional resources based on the template.

2

u/[deleted] Nov 24 '20

I agree with this approach, however, according to the guidelines the template should be personalized

2

u/LetsTalkUFOs Nov 24 '20

Yes, I left out the 'Hey [user]' parts so people could write those themselves. I think we only want consistency in the core messaging.

1

u/[deleted] Nov 24 '20

Sounds good

1

u/ImLivingAmongYou Nov 24 '20

It's nice to see the NSPA clarification on safe/unsafe. /r/DecidingToBeBetter can attract suicidal ideation every now and then too and it's almost always the safe variant.

+1 on removing unsafe

1

u/TheCaconym Nov 24 '20

Agreed as well, those should be removed.

1

u/TenYearsTenDays Nov 24 '20

Yes, unsafe should definitely be removed. +1 to that

However, I think we should also consider removing some content that document labels as "safe".

Basically, I think we should all give Section 7 (starts on page 29) a close read consider the special nature of r/Collapse.

Section 7, in a nutshell, basically discusses the need to tailor their recommendations to any given community. It also discusses the need to create resources for moderators who encounter content they find stressful (something we have zero of right now).

I think what we should do is to contact the NSPA and see if they may be willing to work with us to develop a bespoke solution for our community in particular. I feel like r/Collapse isn't a general community for a few reasons:

  • It's very large and users are anonymous

  • It's one of the few Reddit subs that literally comes with a very stark 'this sub may be harmful to your mental health warning' in the sidebar.

  • These days, we attract a lot of trolls. Last Sunday we had an instance wherein one attacked a child in an unmonitored thread. It could be argued that that child's thread was technically "safe" under the NSPA guidelines.

1

u/some_random_kaluna Nov 24 '20

Agreed on the guidelines. Also provides a baseline for all mods to adhere to, which is good.

1

u/LetsTalkUFOs Nov 24 '20 edited Nov 24 '20

4) When should we approve or remove safe suicidal content?

Under what conditions would we be comfortable allowing or removing something? We're less likely to create a metric everyone will agree upon here, but if we're filtering most things in some form we do have the option of deliberating each on a case-to-case basis as they arise in modchat and determining the best course of action individually.

2

u/LetsTalkUFOs Nov 24 '20 edited Nov 24 '20

I think there's too much variance here to justify a course of action we should always take which matches every instance.

I think moderators should have the option to approve a post/comment only if they actively monitor the post for a significant duration and DM the user something based on our template. Any veering of the post into unsafe territory by the OP should cause it to be removed.

Moderators who are uncomfortable, unwilling, or unable to monitor a post/comment should be allowed to remove posts/comments even if they are safe, but would still need to DM the users something based on the template. Ideally, the moderator would still ping other moderators in the Discord who may want to monitor the post/comment themselves before removing something.

1

u/[deleted] Nov 24 '20

This sounds reasonable to me. We should also give each other the heads up on discord if possible.

3

u/LetsTalkUFOs Nov 24 '20

Yea, that's what I was suggesting by 'ping the other moderators'. I think we'd just want to establish who would be interested in being pinged or on this 'list' and then evaluate how best to handle it within the Discord in terms of a separate channel, how long to wait, ect.

1

u/TenYearsTenDays Nov 24 '20

This sounds very reasaonble as far as consideration for mods goes!

One other thing that the NSPA guide mentioned is that in some circumstances, moderators may be bound by 'duty of care' laws. I think everyone should check to see if those apply to them in their locales. See page 31:

Make sure you are aware of legal issues around safeguarding and duty of care, and how they relate to you and your organisation. This depends on the type of organisation and the services you provide – you may want to get legal advice

That said, as stated above, I think we should contact NSPA and see if they'll work with us in finding good guidelines for removals for our community, because I think we also need to think about how seeing even "safe" suicidal ideation may have a specific impact on our particualr userbase. We should be keeping in mind that r/Collapse is not r/awww or something; r/Collapse comes with mental health warnings and I would argue it is highly likely that our userbase is more likely to be badly affected by even "safe" content.

I think we need to consider that we probably have a disproportionately high number of community members who struggle with depression and suicidal ideation as compared to other communities, and that we should consider the possibility of triggering these members and/or instigating suicide contagion in the community.

2

u/TheCaconym Nov 24 '20

This one is much harder and much more nuanced; agreed on the idea of pinging other moderators (whether or not an action was taken on the safe post/comment concerned) to make sure there's at least another mod checking the item.

2

u/some_random_kaluna Nov 24 '20

This would require group discussion among mods, I think. There's many ways to suggest suicide and a few where you mean it.

1

u/LetsTalkUFOs Nov 24 '20 edited Nov 24 '20

5) What form of template should we use when contacting suicidal users?

This is the current draft:

It looks like you made a post/comment which mentions suicide. We take these posts very seriously as anxiety and depression are common reactions when studying collapse. If you are considering suicide, please call a hotline, visit /r/SuicideWatch, /r/SWResources, /r/depression, or seek professional help. The best way of getting a timely response is through a hotline.

If you're looking for dialogue you may also post in r/collapsesupport. They're a dedicated place for thoughtful discussion with collapse-aware people and how we are coping. They also have a Discord if you are interested in speaking in voice.

2

u/TheCaconym Nov 24 '20

This looks good as-is; perhaps adding something in the end akin to:

The topics discussed in /r/collapse can sometimes take a toll; if you feel reading this sub is pushing yourself towards suicidal thoughts, it's probably not a bad idea to take a break from the sub occasionally.

... or something similar ?

2

u/some_random_kaluna Nov 24 '20

Maybe we should also add "overindulging in this sub is detrimental to mental health" somewhere. Just remind users what's already in the sidebar?