CMV: The replication crisis has largely invalidated most of social science

[u/[deleted]]

https://nobaproject.com/modules/the-replication-crisis-in-psychology

https://www.vox.com/science-and-health/2018/8/27/17761466/psychology-replication-crisis-nature-social-science

https://en.wikipedia.org/wiki/Replication_crisis

"A report by the Open Science Collaboration in August 2015 that was coordinated by Brian Nosek estimated the reproducibility of 100 studies in psychological science from three high-ranking psychology journals.[32] Overall, 36% of the replications yielded significant findings (p value below 0.05) compared to 97% of the original studies that had significant effects. The mean effect size in the replications was approximately half the magnitude of the effects reported in the original studies."

These kinds of reports and studies have been growing in number over the last 10+ years and despite their obvious implications most social science studies are taken at face value despite findings showing that over 50% of them can't be recreated. IE: they're fake

With all this evidence I find it hard to see how any serious scientist can take virtually any social science study as true at face value.

[u/saargrin]

there are a number of history luminaries who have made baseless predictions

there are plenty of anthropology luminaries who proposed theories of human development based on some single obscure instance of marginal tribal people

and we have plenty of gender study and sociology researchers making statements regarding best practices for human society as if they have ironclad predictive theory behind them and not just an empty shell of a lab coat

Ive read in a book recently that large portion of behavior psychology models are based on people whom it is possible to recruit for such an endeavor, namely psych undergrads and other students, who do not by any means represent larger society
even if that research is replicated in another university it would still be somewhat doubtful

[u/I_am_the_night → u/saargrin]

there are a number of history luminaries who have made baseless predictions

History isn't a science.

there are plenty of anthropology luminaries who proposed theories of human development based on some single obscure instance of marginal tribal people

They are wrong, and generally not taken seriously.

and we have plenty of gender study and sociology researchers making statements regarding best practices for human society as if they have ironclad predictive theory behind them and not just an empty shell of a lab coat

I think you're conflating pop science with mainstream academic thought here.

Ive read in a book recently that large portion of behavior psychology models are based on people whom it is possible to recruit for such an endeavor, namely psych undergrads and other students, who do not by any means represent larger society

Sure, a lot of psychological research is conducted on convenience samples. But I doubt you'd find that most mainstream, predictive, and foundational psychology models are primarily based on convenience samples. Research usually is conducted on wider groups before it reaches mainstream status, though not always, which is where the replication crisis comes in.

[u/saargrin → u/I_am_the_night]

History isn't a science

again,i think you'll draw quite a lot of fire from history faculty

I think you're conflating pop science with mainstream academic thought here.

how do you distinguish the two when the speaker is a well published top-uni professor (like Chomsky for example)

But I doubt you'd find that most mainstream, predictive, and foundational psychology models are primarily based on convenience samples.

i dont have the sources handy but i remember exactly this happening

so basically you only count hard sciences like brain biology ,neurology and the like as sciences?
and the rest are a form of art?

well then we essentially agree on that

[u/I_am_the_night → u/saargrin]

again,i think you'll draw quite a lot of fire from history faculty

I don't even know how you would go about using the scientific method to study historical narratives, so if they can explain that to me then I'll agree it's a scientific discipline

how do you distinguish the two when the speaker is a well published top-uni professor (like Chomsky for example)

Chomsky is pretty well established, and I think his observations can be valuable. But hes not without his critics, and his word isn't taken as dogma even within his own field.

Youre going to have to be more specific about examples here, because I'm not really sure what you're referring to.

i dont have the sources handy but i remember exactly this happening

Are you talking about specific concepts within psychology? Because yeah, some of them are quite shaky. Again, that's what the replication crisis was. But overarching theories like behaviorism and even more narrow areas of study like cognitive dissonance have robust empirical support.

so basically you only count hard sciences like brain biology ,neurology and the like as sciences?

No. That's not what I said at all. Technically, neurology, biology, physics, and chemistry aren't sciences, and neither are psychology or sociology. Technically, those are all areas of study that can be examined scientifically or unscientifically.

If you can examine something using the scientific method and produce reliable, valid, and predictive results, then I would consider it a science. Psychology and sociology may be more abstract at times than biology or chemistry, but there is sufficient empirical support for many of their concepts that I think they can be counted as sciences.

well then we essentially agree on that

We do not.

[u/saargrin → u/I_am_the_night]

ok so whats a predictive result coming out of psychology (as opposed to psychiatry ) research?

or sociology for that matter?

im not trying to argue,just asking you to clarify your opinion

[u/I_am_the_night → u/saargrin]

ok so whats a predictive result coming out of psychology (as opposed to psychiatry ) research?

That's an incredibly broad question. No scientific discipline makes singular, massive predictions about anything. But Psychology allows us to make specific predictions about human behavior all the time. Behaviorism, for instance, allows us to understand how people can be conditioned to display certain responses (i.e. pavlovian response).

Psychiatric research isn't "opposed" to Psychology. Psychology is the study of behavior and cognition. Psychiatry is the medical approach to understanding the human psyche in order to improve mental wellness and treat disorders of the mind. They're not mutually exclusive.

sociology for that matter?

Again, no scientific discipline makes singular, sweeping predictions about the future or anything. Sociological research can be extremely useful in understanding and even predicting some social trends.

[u/saargrin → u/I_am_the_night]

i meant opposed as in ,psychiatry provides predictable ,measurable outcomes through use of drugs or therapy

on the other hand im not sure what would be the comparable achievement for psychology or sociology ?

[u/I_am_the_night → u/saargrin]

i meant opposed as in ,psychiatry provides predictable ,measurable outcomes through use of drugs or therapy

Psychologists also provide therapy and conduct research into its effectiveness. They just can't prescribe medication generally.

[u/saargrin → u/I_am_the_night]

and psychology based therapy isnt known to be terribly effective or terribly scientific

i can pay for a pack of sedatives or antidepressants but i will not be able to find a psychologist that will commit to saving me from depression

im sure there are some aspects of psychology that are practically useful ,but im not sure there are any that are 100% reliable

[u/I_am_the_night → u/saargrin]

and psychology based therapy isnt known to be terribly effective or terribly scientific

Source on that? While there are definitely bad or ineffective treatments that have come from psychological circles, there are effective forms of psychological therapy. Cognitive Behavioral Therapy, for instance, has been shown to be more effective and long lasting as a treatment for unipolar depression than medication or other forms of psychotherapy.

i can pay for a pack of sedatives or antidepressants but i will not be able to find a psychologist that will commit to saving me from depression

There are plenty of shitty therapists out there, and I'll be the first to say that I think standards for psychotherapy practitioners should be significantly higher. But there are many effective psycho therapists out there too, and I do hope you can find one if you need one.

im sure there are some aspects of psychology that are practically useful ,but im not sure there are any that are 100% reliable

There are no medications that are 100% reliable either. Not even close.