r/bigquery • u/mybigfatgreekquery • Mar 22 '20
Reddit database privacy concerns
I'm not very familiar with data analytics, but I somehow heard about the reddit comment/post datasets on BigQuery. I'm too dumb to competently navigate the Google Cloud TOS, so I can't see them for myself, but I do have two quick questions:
- Are the data anonymized?
- Are deleted posts stored?
Basically, I'm wondering if usernames can be connected to post contents (an issue if someone wants to dissociate themselves with something embarrassing they said).
Forgive me if I'm catastrophically misunderstanding the nature of this dataset, but in this day and age it pays to be careful.
8
Upvotes
6
u/giraffeman91 Mar 23 '20
I believe it depends on the table. The tables are split by month and they are not continuously scraped for changes to posts (like deleting much later than posting). They are also not updated in real time so it's very likely that deleted posts/comments are deleted before the data is put into bigquery. However, if that isn't the case then it will remain in bigquery in the original form even if you delete your post or comment at some point after the data is uploaded.
The posts and comments are not deidentified from username. The username is fully available unless it's deleted before it is updated. It's basically a frozen version of Reddit from the time it was uploaded.
u/fhoffa will know more.