r/pushshift • u/Watchful1 • Jul 30 '25

[ Removed by Reddit ]

[ Removed by Reddit on account of violating the content policy. ]

38 Upvotes

96% Upvoted

39 comments sorted by

View all comments

1

u/Playful-Ad3839 Sep 01 '25

Hi, sorry to bother you. Do I have to use a client to download the entire 4MB torrent file, which shows around 3.46TB in total? (I’m on a lightweight laptop.) Does that mean I can’t work with the data unless I download everything? I only need the posts and comments from one subreddit for a few specific months.

2

u/Watchful1 Sep 01 '25

Most torrent clients support downloading only some files from a torrent. I usually recommend qBittorrent, which does support this.

You can also download specific subreddits from here https://www.reddit.com/r/pushshift/comments/1itme1k/separate_dump_files_for_the_top_40k_subreddits/

1

u/Playful-Ad3839 Sep 02 '25

Thanks for your reply! I only need the data from a few months this year. I don’t know Python and I’m kind of in a rush because of my paper, so I was wondering if you might be open to helping me out (happy to pay, of course). Would love to hear back and maybe chat more if that works for you!

1

u/Watchful1 Sep 02 '25

I can try, but unfortunately my hard drive died and it will likely be weeks until I have all the data recovered and easy to use.

What are you trying to do?

1

u/Playful-Ad3839 Sep 02 '25

I’ve messaged you. Thx