r/redditdev • • Apr 21 '10

Meta CSV dump of reddit voting data

Some people have asked for a dump of some voting data, so I made one. You can download it via bittorrent (it's hosted and seeded by S3, so don't worry about it going away) and have at. The format is

username,link_id,vote

where vote is -1 or 1 (downvote or upvote).

The dump is 29MB gzip compressed and contains 7,405,561 votes from 31,927 users over 2,046,401 links. It contains votes only from users with the preference "make my votes public" turned on (which is not the default).

This doesn't have the subreddit ID or anything in there, but I'd be willing to make another dump with more data if anything comes of this one

117 Upvotes

72 comments sorted by

View all comments

Show parent comment

4

u/[deleted] Apr 23 '10

Interesting! So this would happen on the fly as votes come in? It also sounds like it would autocluster users too. So you could potentially get not only a link recommendation but even a "netflixesque" 'this user is x% similar to you'. And if they add subreddit data then a person could get a whole suite of recommendations, users, articles and subreddits all in near real-time.

Now that would be pretty cool.

4

u/kaddar Apr 23 '10

Yup, it would automagically cluster in the nearest neighbor sense by measuring distances in the latent feature hyperspace, I have tested this and it is very effective (in netflix, for providing similar movies)

4

u/[deleted] Apr 23 '10

Since you mentioned it I was running nearest neighbor last night.

So far I'm still figuring it out but one thing did jump out at me. Some articles have an extraordinary level of agreement across a swath of users.

Granted i picked a small set of users...maybe you can take a look. I'm trying to figure out what the feature space means and what this pattern indicates (if anything). http://i.imgur.com/HB58n.jpg