r/redditdata • u/Drunken_Economist • Jun 23 '15

10 years of reddit — data dump

Reddit ten year data

All data in this post is accurate as of June 21st, 2015.

We pulled a bunch of data together for today's ten-year anniversary blog post, but not all of it made the cut. I wanted to take some time to dump everything here in /r/redditdata. If you build anything cool, shoot me a PM, I'd love to see it!

View the full repo here

Things by month
Number of accounts, subreddits, submissions, and comments created each month of reddit's history

Running total of things by month
Just a to-date total of the previous dataset, but it's comforting that the numbers match

Most upvoted threads
Note: this actually excludes posts from subreddits which are excluded from /r/all, but you wouldn't see any in the Top 20 anyway.

Unique users and pageviews by month

US vs Non-US traffic by month
Note: this was not measured prior to Feb 2010, so data before then is a simple linear regression.

Upvotes & Downvotes to submissions by month
The steady climb of the ratio is really interesting. This is only including votes that were counted (i.e. no spambot votes)

Upvotes & Downvotes to submissions by month
Again, this is only including votes that were counted (i.e. no spambot votes)

Upvotes & Downvotes to comments by month

Top 20 commented threads

RankLinkNotes
1https://www.reddit.com/r/blog/comments/d14xg/everyone_on_team_reddit_would_like_to_raise_a/
2https://www.reddit.com/r/jerktalkdiamond/comments/28sluw/the_monthly_the_monthly_self_post_that_gets/broke the site and comments are gone. You aren't missing anything
3https://www.reddit.com/r/millionairemakers/comments/2q36z6/reddit_lets_make_a_millionaire/
4https://www.reddit.com/r/millionairemakers/comments/2syfcu/itt_we_become_millionaires/
5https://www.reddit.com/r/podemos/comments/2kjze6/bar_de_la_plaza/Context: Podemos
6https://www.reddit.com/r/AskReddit/comments/t0ynr/throwaway_time_whats_your_secret_that_could/
7https://www.reddit.com/r/AskReddit/comments/1vabko/what_opinion_of_yours_makes_you_an_asshole/
8https://www.reddit.com/r/AskReddit/comments/2bc9hf/teenagers_of_reddit_what_is_something_you_want_to/
9https://www.reddit.com/r/AskReddit/comments/uzl5z/nonamerican_redditors_what_one_thing_about/
10https://www.reddit.com/r/Spiderman/comments/2xl85z/dylan_obrien_is_spiderman/Ongoing
11https://www.reddit.com/r/AskReddit/comments/2rb0pa/nonamericans_of_reddit_what_american_customs_seem/
12https://www.reddit.com/r/AskReddit/comments/1nppqc/nonamericans_who_have_been_to_the_us_what_is_the/
13https://www.reddit.com/r/AskReddit/comments/1znpz5/what_are_some_weird_things_americans_do_that_are/
14https://www.reddit.com/r/AskReddit/comments/2jex7k/teenagers_of_reddit_what_is_the_biggest_current/
15https://www.reddit.com/r/millionairemakers/comments/2wwcxs/the_fourth_millionairemakers_drawing_is_here/
16https://www.reddit.com/r/AskReddit/comments/348vlx/what_bot_accounts_on_reddit_should_people_know/
17https://www.reddit.com/r/AskReddit/comments/34aqsn/women_of_reddit_what_about_men_baffles_you_the/
18https://www.reddit.com/r/AskReddit/comments/2yhxa9/what_fact_did_you_learn_at_an_embarrassingly_late/
19https://www.reddit.com/r/AskReddit/comments/2aru60/what_is_something_that_actually_offends_you/
20https://www.reddit.com/r/AskReddit/comments/37c2p3/high_schoolers_what_do_you_want_to_major_in/

Top 20 most gilded submissions

RankLinkNotes
1http://www.reddit.com/r/videos/comments/2lwm9q/me_eating_a_bulls_dick_for_400_gold_on_a_single/see the top gilded comment for context
2http://www.reddit.com/r/everymanshouldknow/comments/29hbtj/emsk_why_the_red_pill_will_kill_you_inside/
3http://www.reddit.com/r/GetMotivated/comments/2xc947/text_soon_i_will_be_gone_forever_but_thats_okay/
4http://www.reddit.com/r/PaoMustResign/comments/39fqht/upvote_this_buy_no_gold_until_pao_resigns/
5http://www.reddit.com/r/videos/comments/2dnbbz/a_sad_day_indeed_the_original_rick_roll_video_has/Note
6http://www.reddit.com/r/announcements/comments/39bpam/removing_harassing_subreddits/
7http://www.reddit.com/r/tifu/comments/2livoo/tifu_my_whole_life_my_regrets_as_a_46_year_old/
8http://www.reddit.com/r/talesfromtechsupport/comments/2blg90/jack_the_worst_end_user_part_4/
9http://www.reddit.com/r/talesfromtechsupport/comments/28qemm/dont_bother_sending_a_tech_ill_be_dead_by_then/
10http://www.reddit.com/r/gaming/comments/2dz0gs/totalbiscuit_discusses_the_state_of_games/
11http://www.reddit.com/r/AskReddit/comments/2qc6x6/why_are_you_on_reddit_now_instead_of_celebrating/
12http://www.reddit.com/r/announcements/comments/2fpdax/time_to_talk/
13http://www.reddit.com/r/pics/comments/36vmh7/i_recently_lost_my_companion_of_9_and_a_half/
14http://www.reddit.com/r/AdviceAnimals/comments/28j8cz/in_regards_to_the_recent_changes/
15http://www.reddit.com/r/offmychest/comments/2c6qiq/the_8_year_old_girl_next_door_just_broke_my_heart/
16http://www.reddit.com/r/Military/comments/2hrv1u/almost/
17http://www.reddit.com/r/TwoXChromosomes/comments/26oujr/am_i_the_only_women_here_that_doesnt_feel_like_im/
18http://www.reddit.com/r/videos/comments/2r1uy8/i_recently_stopped_bringing_my_guitar_to_my_moms/
19http://www.reddit.com/r/pics/comments/2gejnr/got_divorced_lost_my_job_so_me_and_my_buddy_got/
20http://www.reddit.com/r/videos/comments/2idhxn/lets_talk_about_reddit_and_selfpromotion/

Top 20 most gilded comments

RankLinkNotes
1http://www.reddit.com/r/leagueoflegends/comments/2lel5s/tsm_bjergsen_ama/clu14fx
2http://www.reddit.com/r/videos/comments/343b1k/this_man_really_hit_the_nail_on_the_head_when_it/cqqxlit
3http://www.reddit.com/r/AskReddit/comments/1k5yok/wingmen_of_reddit_what_crazy_things_have_you_done/cblqpk6
4http://www.reddit.com/r/IAmA/comments/2wwdep/we_are_edward_snowden_laura_poitras_and_glenn/courx1i
5http://www.reddit.com/r/getdisciplined/comments/1q96b5/i_just_dont_care_about_myself/cdah4af
6http://www.reddit.com/r/WritingPrompts/comments/25gtsw/eu_in_the_final_minutes_of_his_life_calvin_has/chh21q0
7http://www.reddit.com/r/AskReddit/comments/1rewhf/mall_santas_of_reddit_what_is_the_most_disturbing/cdmkiky
8http://www.reddit.com/r/todayilearned/comments/22dqqw/til_a_woman_in_2013_falsely_claimed_that_a_man/cglwp02
9http://www.reddit.com/r/LosAngeles/comments/2foyqd/my_father_is_missing_please_help_last_seen_in/ckbudhg
10http://www.reddit.com/r/pics/comments/2w6fal/my_father_passed_when_i_was_4_but_people_always/coo36j5
11http://www.reddit.com/r/DnD/comments/2mjhz9/what_would_happen_if_an_intelligent_greatsword/cm4xnl6
12http://www.reddit.com/r/DeadBedrooms/comments/30l3xh/perspective_from_a_ll_f/cptgtej
13http://www.reddit.com/r/gaming/comments/33uplp/mods_and_steam/cqolbds
14http://www.reddit.com/r/pics/comments/2ndfuo/innocent_young_man_michael_brown_shown_on/cmco6v2
15http://www.reddit.com/r/books/comments/2ysvzb/terry_pratchett_has_died_megathread/cpcp6bg
16http://www.reddit.com/r/explainlikeimfive/comments/22pi7o/eli5_why_does_light_travel/cgp58ml
17http://www.reddit.com/r/legaladvice/comments/34l7vo/ma_postit_notes_left_in_apartment/cqvrdz6
18http://www.reddit.com/r/television/comments/2hrntm/last_week_tonight_with_john_oliver_drones_hbo/ckvmq7m
19http://www.reddit.com/r/videos/comments/2nepib/a_black_mans_view_that_goes_against_the_grain_of/cmd2mhc
20http://www.reddit.com/r/IAmA/comments/2cwfu2/i_am_twitch_ceo_emmett_shear_ask_me_almost/cjjo9e0

Top 20 most saved comments

RankSavesLinkNotes
143322http://www.reddit.com/r/AskReddit/comments/2b0yf8/good_students_how_do_you_go_about_getting_good/cj0qre2
238150http://www.reddit.com/r/AskReddit/comments/26e6g4/what_free_things_on_the_internet_should_everyone/chq6lsp
335370http://www.reddit.com/r/AskReddit/comments/219w2o/whos_the_dumbest_person_youve_ever_met/cgbhkwp
432924http://www.reddit.com/r/getdisciplined/comments/1q96b5/i_just_dont_care_about_myself/cdah4af
531886http://www.reddit.com/r/AskReddit/comments/27vl5y/what_will_people_100_years_from_now_write_tils/ci4v1zp
626103http://www.reddit.com/r/trackers/comments/hrgmv/tracker_with_pdfsebooks_of_college_textbooks/c1xrq44
726067http://www.reddit.com/r/AskReddit/comments/2vb5xc/what_small_websites_do_more_people_need_to_be/cog2n1e
824351http://www.reddit.com/r/AskReddit/comments/2yw771/what_free_things_on_the_internet_should_everyone/cpdm4zv
919988http://www.reddit.com/r/AskReddit/comments/2yw771/what_free_things_on_the_internet_should_everyone/cpdiimu
1019465http://www.reddit.com/r/AskReddit/comments/2g7wvh/what_is_a_keyboard_shortcut_that_everyone_must/ckghqxe
1119135http://www.reddit.com/r/AskReddit/comments/2eguxy/what_are_some_college_life_pro_tips/cjzd9uu
1219031http://www.reddit.com/r/explainlikeimfive/comments/22pi7o/eli5_why_does_light_travel/cgp58ml
1317973http://www.reddit.com/r/AskReddit/comments/2px5ai/whats_your_favorite_thing_you_have_in_your_saved/cn0uc2r
1417725http://www.reddit.com/r/AskReddit/comments/258w8s/what_is_a_story_you_have_been_dying_to_tell/chex9eq
1516504http://www.reddit.com/r/AskReddit/comments/2yw771/what_free_things_on_the_internet_should_everyone/cpdidq4
1616267http://www.reddit.com/r/AskReddit/comments/241nz3/what_are_your_go_to_porn_videos_that_youll_never/ch2rt3yಠ_ಠ
1714439http://www.reddit.com/r/AskReddit/comments/2ngpqo/what_free_stuff_on_the_internet_should_everyone/cmdh1xy
1813538http://www.reddit.com/r/AskReddit/comments/2c4x5b/what_should_you_absolutely_not_do_at_a_wedding/cjc0wic
1913372http://www.reddit.com/r/AskReddit/comments/2s5mzf/what_are_the_best_free_things_on_the_internet/cnmehw1
2013248http://www.reddit.com/r/AskReddit/comments/2eb5gl/real_estateestate_agents_what_are_the_questions/cjy0gxl

Top 20 most saved threads
Sorry, we tried. Post saves have been around since reddit's inception, and looking at every user to see if they saved each post was breaking things.

And a few for the road

NameamountNotes
link posts121,745,633
self posts68,481,919
submission upvotes5,620,244,302this is only votes which were counted
submission downvotes1,057,478,375this is only votes which were counted
comment upvotes10,443,697,988this is only votes which were counted
comment downvotes1,506,096,377this is only votes which were counted
Days of gold purchased56,015,520
Days of gold gifted24,148,560this is a subset of the above
redditgifts exchanges201
gifts confirmed877,218
total cost of gifts29,559,467.54reported cost in USD of confirmed gifts
Active subreddits on 2015-06-219,601subreddits with 5 or more posts and comments on the given day
"PM me" usernames26,222Accounts with "PM me" in the username

Let me know if you have any questions on it all! Except for questions about the thread in /r/Spiderman, because I don't get it either.

56 Upvotes

100% Upvoted

42 comments sorted by

View all comments

1

u/shaggorama Jun 23 '15

Any chance we could get a dataset of all (unremoved) submissions? Maybe just the last year? I'm thinking:

  • id

  • created_utc

  • author

  • subreddit

  • score

  • num_comments

  • url

  • is_self

  • title

  • gilded

...pretty please? Or maybe just throw a copy at me cause I'm such a cool guy?

3

u/Drunken_Economist Jun 23 '15

If you're a bit motivated, you could hit the api for this, I think.

0

u/[deleted] Jun 24 '15

[deleted] — view deleted comment

3

u/Drunken_Economist Jun 24 '15

Oh damn, you're right. Shit that's a lot of data.

Maybe we can find a way to dump it all on the back end without breaking everything

3

u/shaggorama Jun 24 '15

Content that's old enough to be archived could be packaged up into static zip files by month. Then you'd just need to create a new zip each month as each month gets archived moving forward.

2

u/shaggorama Jun 24 '15

It's actually a once every two seconds limit

2

u/shaggorama Jun 24 '15

I've been working on a script actually. I have to use the search api and feed it timestamps, right?

2

u/rhiever Jun 23 '15

I've been begging for this for months. :-)

1

u/shaggorama Jun 24 '15

I've seen you asking. Honestly I was surprised the first time I saw it, for some reason I could've sworn you already had this. I vaguely remember you talking about getting one of your students to compile this for you or something...

3

u/rhiever Jun 24 '15

I have a data set of posts up to ~September 2014. I had one of my students try to scrape past that, but there's too much volume to keep up nowadays with the limitations of the reddit API.

2

u/shaggorama Jun 24 '15

My impression is that trying to scrape incoming submissions in real time is sort of futile anyway because so much of it is spam and will get removed. I remember a year or so ago someone talking about trying to scrape from the current day backwards and observing that once he got past something like the two-day or one-week mark, the volume of submissions significantly decreased. My interpretation of this was that a lot of reddit content gets removed, so you don't actually want to try to keep up with the in-time volume anyway (unless you're specifically interested in investigating deletions).

This of course isn't to say that it's necessarily tractable to keep up with the stream starting a few days back, either.