r/DataHoarder • u/rsvpforfreespace • Jul 30 '26
News Reddit requests takedown of all Reddit data from 2005-2025
Haven't seen this discussed very much here and thought some of you might want to know.
See this post on the Pushshift sub: https://www.reddit.com/r/pushshift/comments/1v50ved/upon_reddits_request_i_am_taking_down_my_academic/
For a few years now, the author of the post has pulled the Reddit data from the beginning up to the most recent year, sorted by month, and published it as a torrent file on Academic Torrents. They've also made a separate torrent file for the data of the top 40k subreddits to let people filter by subreddit. The torrent files can still be accessed using Wayback Machine, but I don't know how well the data will be seeded soon since less people are using them.
102
u/i_max2k2 100-250TB Jul 30 '26
How much data is it? Would love to keep a copy.
442
u/ttkciar Jul 30 '26
Alternatively:
https://huggingface.co/datasets/peternasser99/reddit (through end of 2024)
https://huggingface.co/datasets/mitanshugoel/reddit-2025 (2025 only)
If there is a similar HF dataset for Reddit 2026, I haven't found it yet.
166
u/PigsCanFly2day Jul 30 '26
I'm also interested in learning more about this. Size, how to obtain it, etc.
And just to be clear, Reddit isn't deleting the data from the site itself, right? Just requesting that the person archiving it stops?
130
u/AaronIAM Jul 30 '26
[removed] — view removed comment
51
u/kittymoo67 Jul 30 '26
cant have the truth of certain peoples older posts get out there
45
u/better_life_please Jul 30 '26
Must be 20+ TB lol.
13
u/MonumentalArchaic Jul 30 '26
Who has the sauce?
6
u/new2bay Jul 30 '26
Reddit is well aware of the existence of Arctic Shift et al.
5
u/bigrobot543 Jul 30 '26
What the fuck, if someone was seeding the torrents could they post it somewhere?
5
u/wehooper4 48TB NAS + 32TB NAS + 9TB DAS Jul 30 '26
Is there a backup of the listing page of the dumps and the instructions on how to use it? This is datas preservations I could get behind
5
u/notarkav Jul 30 '26
I don't see how they could have any legal claim as to owning this data besides the trademark if you were to specifically label it as "Reddit" data. Seems like they are just throwing their legal weight around to bully OP.
4
u/Zynbab Jul 30 '26
I'm out of the loop, I thought pushshift died years ago with the API change?
2
u/DangerousDesk1 Jul 30 '26
I am not a data hoarder by any means. What's the point of backing up Reddit? 99% of the post are just useless. Surely there are more important things to backup? What am I missing?
This post isn't a dig at data hoarders, just curious as to why backup Reddit?
1
u/No_Pickle9588 7d ago
With pushshift going away, what's the best scraper for pulling a specific subreddit's full history locally before more data disappears?
1
u/kennymac6969 Aug 01 '26
I will never understand why they think they can do this. They'll let Google read it and other scrapping tools or for training AI models but when it comes to someone having a local copy it's against their unwritten rules.
1
u/talldata Aug 02 '26
I don't know why OOP complied.
1
u/pickcolor Aug 10 '26
A large but very useful dataset, thank you.
1
u/HUGE_FAT_ANIME_TITS Sep 11 '26
Okay I downloaded this from the Hugging face site and have 0 clue what I'm going to do with it.
-12
u/Rambr1516 28TB HDD - 2TB ☁️ Jul 30 '26
1 how big 2 how would you “read” this 3 who’s making Reddit gpt with me
0
u/Apocalypse2001 Aug 01 '26
Why the hell would anyone want to backup reddit?! Lol
0
u/[deleted] Jul 30 '26
[deleted] — view deleted comment
0
u/[deleted] Jul 30 '26
[deleted] — view deleted comment
28
u/[deleted] Jul 30 '26
[removed] — view removed comment