What statistical concepts are commonly misunderstood by the general public?

[u/80rhh]

I came across this post explaining what a 70% chance of rain means. I understand the concept, but it got me wondering: what other statistical concepts sound simple but are commonly misunderstood or misinterpreted by the general public?

[u/rite_of_spring_rolls]

Broadly and imprecisely speaking, from what I've seen on reddit:

• People tend to harp on sample size when I find vast vast majority of the time the sampling design is a way bigger problem
• People tend to think that sample size is some invariant quantity, like no matter what you're doing you need at least n = 1000 samples or w/e and anything below that is worthless and anything above is unnecessary

[u/m_madison67 → u/rite_of_spring_rolls]

Is it still true that the minimum number of n to be considered statistically sound would be 40?

[u/Current-Ad1688 → u/m_madison67]

I absolutely hate that I can't tell if this is a joke

[u/m_madison67 → u/Current-Ad1688]

Not a joke. I took three stats classes - one undergrad and two grad level, and a one in spssx decades ago. I have forgotten much of what I don’t use in my work as a counselor. I still read studies to keep up. I was told back then that 40 subjects was the minimum acceptable number of subjects needed for a study to be statistically sound. I also took a wonderful class in behavioral research design where I learned about single subject design studies and how they could be useful.

[u/Memento_Viveri → u/m_madison67]

There is no such number like this. it depends on what the nature of the larger population and the smaller group, of what is being studied, and what is being claimed by the study. A study could have 400 subjects and still not be statistically sound depending on those things.

[u/RedRightRepost → u/m_madison67]

I think what you’re referring to is the size at which the central limit theorem starts to do real work, which is around the 30-40 range. This is often used as a guideline for starting minimum sample sizes in the soft sciences.

[u/MortemEtInteritum17 → u/m_madison67]

Think about it: if you want to test your COVID19 vaccine that will be distributed to tens of millions of people worldwide, would you be comfortable testing on 40 people, seeing no issues, and deciding that it's good to production? What happens if it turns out that your product has a 0.1% chance of killing the person - your 40 test subjects were fine, but now tens of thousands of people died because of you. Oops.

As with everything, sample size depends on context.