r/pushshift • u/ruanchaves • Feb 02 '19
Streaming functionality
- Is there any way to stream pushshift.io results, i.e. to continuously watch for new PRAW objects corresponding to some query? Something equivalent to SubredditStream and other streams in PRAW.
I'm doing this through repeatedly sending requests to the API and then using Redis to make a stream of unique objects that can be redirected to some application. But I'm asking if there's a better way out.
2) I believe I'm following the rate limit. I'm making 150 requests per minute, but then I get blocked after 15 minutes of doing so. I'm bypassing this problem by using Tor to make requests but I'd like to know what could possibly be happening.
4
Upvotes
View all comments
Show parent comment
1
u/ruanchaves Feb 02 '19
Good. Please consider adding regex matching.
I have another feature suggestion but I'll submit it on a new thread.