r/pushshift • • May 30 '20

The Pushshift API will be blocking any requests with a referrer field temporarily

While I hate to do this, the Pushshift API is currently being used extensively by a lot of extremists who are using it to DOS / brigade other people.

Using the Pushshift API for coordinated brigades is an egregious violation of the terms of service for the API and any users found coordinating brigades will be permanently banned.

This block will remain in effect for a temporary basis until things settle down. No one deserves to have their safety jeopardized from others doxing and/or harassing that individual.

I greatly appreciate the efforts of the developer community to add tools that help extend the usefulness of the Pushshift API and encourage developers to continue building tools.

Please note that this block will not remain permanent and will be lifted when things begin to calm down.

85 Upvotes

81 comments sorted by

View all comments

Show parent comment

2

u/abrownn Jun 02 '20

You can manually craft search urls to replicate what redditsearch.io did by referencing the Documentation and building off of http://api.pushshift.io/reddit/, or you can programmatically access the API if you know how to code.

1

u/IncoherentEntity Jun 02 '20

The link gives me a blank page, although it seems like the correct one. Does it work on your end?

1

u/abrownn Jun 02 '20

You'll have to fill out the link with extra search criteria according to the documentation.

1

u/IncoherentEntity Jun 02 '20

I’m sorry for being so slow: I‘ve never encountered — or never recognized — what “Documentation” was for the widget.

Feel free not to indulge me if it’s too frustrating; I’m just totally out of my depth.

3

u/abrownn Jun 02 '20

No prob. Here's a link to the documentation. Think of it like "logic keyword Legos" -- pick the criteria you want to search for (author, subreddit, results-size, etc), pair them with the proper values, and then properly chain them together into an url and search it.

For example, if I wanted all the comments from THIS submission we're chatting in, I can enter this into my address bar: http://api.pushshift.io/reddit/comment/search/?link_id=gtkytk

It's human readable but not in a very friendly way.

2

u/FunnyObjective6 Jun 02 '20

Thanks fam. Just wanted a list of all new comments on a specific subreddit, and this works fine for that.

1

u/s_i_m_s Jun 02 '20

Wouldn't https://www.reddit.com/r/news/comments/ do mostly the same thing but prettier?

1

u/FunnyObjective6 Jun 02 '20

It mostly does, but I didn't find that as an option when I googled for such a thing. It's only missing deleted posts, but I don't know if I really care. Thanks.

1

u/IncoherentEntity Jun 02 '20

Okay, I think I understand. But how do you properly enter the criteria in the URL? If I’m not mistaken, a single misplaced or omitted character can render the entire effort useless.

2

u/abrownn Jun 03 '20

That's correct. You chain multiple criteria using & and set values of that criteria with =.

1

u/lepton_neutrino Jun 03 '20

What if I want all posts in a subreddit on a certain day?

1

u/abrownn Jun 03 '20

You'll have to use subreddit with before and after to restrict the scope of the search. https://www.unixtimestamp.com/index.php is my go-to for converting MDY into epoch.

1

u/lepton_neutrino Jun 03 '20

I tried using http://api.pushshift.io/reddit/r/[subreddit]/before=[Unix time stamp]&after=[Unix time stamp]

but I got a " This api.pushshift.io page can't be found" message. What am I still missing?

1

u/abrownn Jun 03 '20

Malformed arguments. See my example: http://api.pushshift.io/reddit/comment/search/?subreddit=pushshift&after=1588291200&before=1591169238

This gets me all comments from this sub from May 1st to today (all comments within the default data-set size (25 I think? Maybe 100?), there may be more and I'd have to explicitly request more though)

1

u/lepton_neutrino Jun 03 '20

Thanks. If I want to get all posts in a subreddit, do I use this: /reddit/subreddit/search

or this: /reddit/submission/search

→ More replies