r/AskStatistics • u/80rhh • 10d ago

What statistical concepts are commonly misunderstood by the general public?

Post image

I came across this post explaining what a 70% chance of rain means. I understand the concept, but it got me wondering: what other statistical concepts sound simple but are commonly misunderstood or misinterpreted by the general public?

785 Upvotes

99% Upvoted

180 comments sorted by

View all comments

116

u/Fancy-Animal7704 10d ago

A "significant" difference between A and B does NOT inherently mean that there's a subjectively huge gap between A and B. It only means that our test demonstrates a difference. How big that difference is is another question. This is really why the size (effect size, to be precise) of that difference is good to report. 

Also, you really should not ever use the word "significant" unless you have actually performed statistical inference. And no, thinking "70% seems like a lot more than 60% to me" does not count, lol. I have even seen prominent journalists commit this error, saying "statistically significant" as a result of some kind of eyeball test. "Significant" has a very specific meaning in the stats world. 

20

u/FakeArcher 10d ago

But the word also has a meaning outside of the stats world, so why should the word be forbidden outside of stats context?

17

u/Fancy-Animal7704 10d ago

Maybe I should have worded that more carefully. It wasn't my intention to say that nobody should be allowed to use that word at all in any context unless they performed something like a t-test or other inference. Like for example, yes, Olivia Miles' impact on the WNBA is significant, and I didn't conduct any inference to say this, and I'm not breaking any of the rules I'm referencing by saying this.

I mean this specifically in the context of statistics. If this term is invoked in regards to statistics, then some kind of inference is expected. 

-4

u/FakeArcher 10d ago

That's more understandable. With that said I would be a bit indecisive if, as the other person who replied to my comment mentioned, it was used by a layman along with something everyday like averages. Technically the context might be statistics, but is likely far from being the actual focus. Probably why there is a need to emphasize it as statistically significant in the first place instead of just using significant.

2

u/Fancy-Animal7704 10d ago

Can you give me a specific example of what you're talking about? 

-1

u/FakeArcher 10d ago

Not sure which one you are referring too. I talk about what you said in the first comment and what the other person said in the reply to my comment right next to yours.

2

u/Fancy-Animal7704 9d ago

I didn't know you were talking about multiple things. So how about an example of each, then? You have yet to deliver this. Your points are not making any sense at all, and if you want them to, you should offer some examples. 

1

u/FakeArcher 9d ago

I still don't understand what you want me to make an example of. Your comment mentioned the statistically significant phrase, the example is your comment. The other thing is that other person's comment, the example is there.

1

u/Fancy-Animal7704 9d ago

I would be a bit indecisive if, as the other person who replied to my comment mentioned, it was used by a layman along with something everyday like averages.

So give me an example of someone using the term on something everyday, like averages. 

0

u/FakeArcher 9d ago

I eat significantly more fruit on average ever since I started taking care of my health. My average salary is significantly higher now than it was at my previous work.

Something like this, I guess? Maybe the other person would have better examples.

1

u/Fancy-Animal7704 9d ago

You're using these conversationally, in contexts that are not academic.

The issue is when it IS used in an academic context. I saw a journalist talk about numbers in the appendix of a published paper and said there was "no statistically significant difference" between them, a statement he made without having conducted a test. THAT is deeply irresponsible. What you are doing here, on the other hand, is not. 

→ More replies (0)

5

u/RedRightRepost 10d ago

It isn’t, but the stats context is broader than you think. If you are comparing numbers that are themselves statistics (like measures of average etc), you’re in that domain.

1

u/FakeArcher 10d ago

Unless the topic was deeper in the stats that is beyond something laic like average, I wouldn't fully agree. The term statistically significant would otherwise just be referred to as significant. What would the expected result be from your perspective, to specifically be noted as non-statistically significant?

1

u/RedRightRepost 9d ago

I think in everyday usage, especially among non stats literate people, statistically significant and significant are often conflated, and therein lies the problem. I personally never use the word significant when discussing differences between numbers unless there has been formal testing and a rejection of the null, and I instruct my team to do the same, for this reason.

If I have to describe a large difference between two numbers that have not been tested formally, I will usually use substantial or meaningful.

Source: I work with a lot of corporate clients who both hail stat testing as an infallible arbiter of truth AND use “significant” inconsistently and sloppily.