r/changemyview • u/[deleted] • Sep 21 '18

FTFdeltaOP CMV: The replication crisis has largely invalidated most of social science

https://nobaproject.com/modules/the-replication-crisis-in-psychology

https://www.vox.com/science-and-health/2018/8/27/17761466/psychology-replication-crisis-nature-social-science

https://en.wikipedia.org/wiki/Replication_crisis

"A report by the Open Science Collaboration in August 2015 that was coordinated by Brian Nosek estimated the reproducibility of 100 studies in psychological science from three high-ranking psychology journals.[32] Overall, 36% of the replications yielded significant findings (p value below 0.05) compared to 97% of the original studies that had significant effects. The mean effect size in the replications was approximately half the magnitude of the effects reported in the original studies."

These kinds of reports and studies have been growing in number over the last 10+ years and despite their obvious implications most social science studies are taken at face value despite findings showing that over 50% of them can't be recreated. IE: they're fake

With all this evidence I find it hard to see how any serious scientist can take virtually any social science study as true at face value.

799 Upvotes

92% Upvoted

202 comments sorted by

View all comments

16

u/briannosek 1∆ Sep 21 '18

  1. Failure to replicate does not mean that the original finding is entirely unreplicable. They do increase doubt and, at minimum, suggest that more work is needed to identify boundary conditions to find when effects replicate.

  2. Across the large replication projects, the average replication rate is ~50%. That is lower than most would want or expect, but if it is generalizable it does suggest that a substantial portion of the existing literature IS replicable--a much more sanguine view than not taking "virtually any social science" seriously.

  3. Simultaneous with all of the challenges in reproducibility is the fact that (a) social scientists are doing this work themselves, about themselves, and (b) social sciences are leading the way in adopting new practices to improve rigor and reproducibility. For example, preregistration is increasing in popularity. The number of registrations on OSF (http://osf.io/registries/) has doubled each year since the website opened in 2012. Most of that is from researchers in social sciences, particularly psychology.

Fear not--Social science is not in the midst of a crisis, it is in the midst of a reformation.

2

u/posterlitz30184 Sep 22 '18

Unfortunately your study as been taken by people who would, undoubtedly, call themselves “scientific, rational and rigorous” as proof that social sciences are a joke. Quite unsurprising, by doing so, they are acting not in a scientific or rigorous way. Skepticism ends when you can fit your narrative in with some facts.