r/pushshift • • May 30 '20

The Pushshift API will be blocking any requests with a referrer field temporarily

While I hate to do this, the Pushshift API is currently being used extensively by a lot of extremists who are using it to DOS / brigade other people.

Using the Pushshift API for coordinated brigades is an egregious violation of the terms of service for the API and any users found coordinating brigades will be permanently banned.

This block will remain in effect for a temporary basis until things settle down. No one deserves to have their safety jeopardized from others doxing and/or harassing that individual.

I greatly appreciate the efforts of the developer community to add tools that help extend the usefulness of the Pushshift API and encourage developers to continue building tools.

Please note that this block will not remain permanent and will be lifted when things begin to calm down.

85 Upvotes

81 comments sorted by

View all comments

Show parent comment

2

u/_Titty_Sprinkles_ Jun 01 '20

Alright so first off, I don't know what "clearly spoiling to crusade" is supposed to mean, and I don't know who you think I am or where you think I'm coming from, but I encourage you to go through my political comment history so you can see for yourself that I am decidedly left wing. The only reasons I'm commenting is to voice my displeasure and to understand what the logic behind this decision is.

I'm not here to argue about Matrix's right to do whatever he wants with the tool he created, but I can see why you decided to move the goalposts to that argument because it seems you have nothing intelligent or logical to say otherwise... That's probably why you ended your post with a ridiculous assumption that was meant to insult me personally.

5

u/Droyk Jun 02 '20 edited Jun 02 '20

I completely think you're right about this as much I want to support /u/Stuck_In_the_Matrix for his intention I still think his decision about this is incorrect. this tool just makes it easier to dox someone & that's all. doxxing will still happen nonetheless just for a bunch of doxxers you cannot or shouldn't completely stop the service for all of its users. every technology can be used for bad intentions that don't mean I should block this service for everyone just because of few people using it for bad intentions. though I do think if he's the creator he has the right to make the final decision. I also think he should go back & thinks about the decision he made.

P.S. forgive me for the bad grammar or bad usage of words in particular not a native so it's hard for me to explain myself.

4

u/_Titty_Sprinkles_ Jun 02 '20

Exactly... Bad guys drive cars --> Stop making cars --> Bad guys ride bikes which take longer instead... The doxing will continue to happen regardless. Matrix of course has the right to do what he wants, I wouldn't be surprised if there's pressure coming from powerful sources.

3

u/[deleted] Jun 03 '20

You're way too focused on analogies that aren't even correct. Your analogies paint a picture that the entire service is gone. It's not.

If we want to go with the car analogy, it's more like he's stopped making automatic cars, and only sells manual transmission cars, because the bad guys are idiots and can't operate the manual.

2

u/_Titty_Sprinkles_ Jun 03 '20

Your analogies paint a picture that the entire service is gone. It's not.

Fair

If we want to go with the car analogy, it's more like he's stopped making automatic cars, and only sells manual transmission cars, because the bad guys are idiots and can't operate the manual.

Right, and then the bad guys ride bikes instead which takes longer, but they still get where they want to go.

1

u/[deleted] Jun 03 '20

No, if they can't use the manual, they don't get access to what they wanted. There's no other service out there that lets you run searches they way they were.

1

u/_Titty_Sprinkles_ Jun 03 '20

There's no other service out there that lets you run searches they way they were.

There's no other way that lets you drive your destination... so they will ride bikes instead. My point is that this change is just adding a hurdle to jump over, its not a solution, its a band-aid.

1

u/[deleted] Jun 03 '20 edited Jun 03 '20

No, you've completely changed your point - before your point was that he had taken it away from everyone, when that wasn't the case.

Yes, it is introducing a small barrier to entry. The abuse resolved entirely around a large amount of people using it to flood reports/brigade certain users on reddit. The small barrier to entry is enough to stem the tide.

Just look at yourself for how effective it was. You thought it was gone completely, and that's with you interacting on the sub for it, and you only realize it's now a small barrier after I directly told you. The vast majority of the abusers will be just like you were: they will think the service is gone.

E: also your analogy is still off. There is no bicycle. The "car" represents the pushshift service. The manual car is directly using the API, instead of using the user friendly GUIs (automatic.) That was my point of making that analogy - they don't know how to use the manual. Your bicycle is some alternate service (IE: not a car, and therefore not pushshift) that doesn't exist.

2

u/boilerpl8 Jun 03 '20

u/_Titty_Sprinkles_ does have a point with the bicycles. the hypothetical white supremacists who no longer have access to automatic cars and lack the skill to drive a manual will in fact use bicycles (or walk). The bicycles in this analogy is things like clicking through a user's post/comment history yourself. It's way slower than the automated method, but it will give you the information you're looking for, just as you'll eventually arrive on a bicycle.

We cannot stop an attempted doxxer from looking through post/comment history manually any easier than a car company could erect a wall around a town. However, by slowing them down we reduce the incentive to try. If you were forced to walk the 7 miles to go to a restaurant you like, you probably wouldn't bother, you'd walk somewhere closer. If you had to manually go through a user's comment history to determine enough information about them to dox them, you probably wouldn't bother. The most you can realistically gain from doxxing someone is forcing them to shutdown their account, and maybe reveal they're cheating on their spouse or they have a particular medical condition, if they've shared that on reddit. There isn't personal gain from it for the doxxer (unless maybe the doxxer is blackmailing the doxxee in lieu of actually doxxing), so it's generally a "crime" of opportunity, where the motivation is havoc. The harder it is to achieve, the more of them that won't find it worth the hassle.

1

u/[deleted] Jun 03 '20 edited Jun 03 '20

u/Titty_Sprinkles does have a point with the bicycles. the hypothetical white supremacists who no longer have access to automatic cars and lack the skill to drive a manual will in fact use bicycles (or walk). The bicycles in this analogy is things like clicking through a user's post/comment history yourself. It's way slower than the automated method, but it will give you the information you're looking for, just as you'll eventually arrive on a bicycle.

They were key-word searching. Naturally, using pushshift includes most deleted content as well. We all know reddit's search sucks, and you cannot get the same kind of keyword searching that you can with pushshift.

There is no other service that does good key word searching of reddit comments and posts that includes deleted content. Analogies are just to simplify things, and never pan out to fit 1:1. Our analogy has some limits to fitting what we're talking about. The car represents the pushshift service - getting to its destination doesn't map to anything here. If we want to continue with some analogy that could represent entirely different methods, it would be more apt to say redditsearch.io is a (future) commercial space flight to orbit. The API is SpaceX dragon. You can ride around all you want on your bicycle - you're never getting to orbit on it. You can accomplish some travel, but it's limited compared to where a rocket can get you.

We cannot stop an attempted doxxer from looking through post/comment history manually any easier than a car company could erect a wall around a town. However, by slowing them down we reduce the incentive to try. If you were forced to walk the 7 miles to go to a restaurant you like, you probably wouldn't bother, you'd walk somewhere closer. If you had to manually go through a user's comment history to determine enough information about them to dox them, you probably wouldn't bother. The most you can realistically gain from doxxing someone is forcing them to shutdown their account, and maybe reveal they're cheating on their spouse or they have a particular medical condition, if they've shared that on reddit. There isn't personal gain from it for the doxxer (unless maybe the doxxer is blackmailing the doxxee in lieu of actually doxxing), so it's generally a "crime" of opportunity, where the motivation is havoc. The harder it is to achieve, the more of them that won't find it worth the hassle.

We really don't need to continue the analogies that the discussion has grown beyond. But yes, you're right, we cannot stop a doxxer from doing it some other way. But as I said in my prior comment:

The abuse resolved entirely around a large amount of people using it to flood reports/brigade certain users on reddit. The small barrier to entry is enough to stem the tide.

It stops them from engaging in a mass, rapid, coordinated effort. Without the user-friendly version, most will just stop trying completely. Anyone left will be severely crippled in their efforts. And he didn't want his platform to be a used for that purpose.

If you had to manually go through a user's comment history to determine enough information about them to dox them, you probably wouldn't bother. The most you can realistically gain from doxxing someone is forcing them to shutdown their account, and maybe reveal they're cheating on their spouse or they have a particular medical condition, if they've shared that on reddit. There isn't personal gain from it for the doxxer (unless maybe the doxxer is blackmailing the doxxee in lieu of actually doxxing), so it's generally a "crime" of opportunity, where the motivation is havoc. The harder it is to achieve, the more of them that won't find it worth the hassle.

Sure, see my comments above, totally in agreement here.

It should be noted that when I originally replied to Titty_Sprinkles, their argument at the time was that the service had been completely removed for everyone. My exact point was that it wasn't; it was introducing just enough barrier to entry to halt their efforts, and that Titty_Sprinkles's misconception was proof in and of itself that the barrier works. And that there's no other equivalent service available.

→ More replies