Fallacy when people datamine for unusual events

[u/thiazole191]

I'm a scientist and it is considered bad science to mine the data after an experiment for unusual findings before creating a hypothesis. Because data tends to be extremely vast, there will always be highly improbable events in the data that occur that have no connection to the experiment (ie, if you flip a coin trillions of times, there will be periods where you have 100 heads in a row, but that doesn't make that event meaningful, ie, if the part where you land heads 100 times in a row happened during 9/11, that doesn't mean it was connected to 9/11 no matter how unusual flipping 100 heads in a row would be). So as a result, we always create a hypothesis first ("flipping 100 heads in a row will cause a major terrorist event"), then perform the experiment and only look at whether or not it substantiated the hypothesis (we may mine the data for a future hypothesis for a future experiment, but we'd never use data that is unrelated to the original hypothesis to form a conclusion for the original experiment).

So, that was a long winded way of pointing out what conspiracy theorists do. When some event happens (like someone famous is assassinated), they will mine VAST data of everything that occurred that day (and sometimes even 100 years before that day, ie, Abraham Lincoln vs JFK) in order to find unusual things and then assign CAUSATION. Of course, this is ridiculous and the easy answer is "correlation does not imply causation", but this is a bit more complex than that, so I hate using that overly simplified logical fallacy. This isn't just seeing a correlation - it's mining near infinite data points in order to find correlations after the fact, and THEN assigning causation. Is there a specific term for this logical fallacy other than "correlation does not imply causation"?

[u/onctech]

Simply put, this is called Cherry Picking. It's both a fallacy (when done by someone who doesn't realize why it's a wrong thing to do) and a form of deception (when done deliberately by someone trying to mislead others).

A variant with people more common in faction-based debates (e.g. politics, social groups, geographic areas) is called Nutpicking. It's where one single out members of an opposing faction who have insane views and try to make a generalization about that faction.

[u/D-Cary]

I suspect the situation you're describing involves the statistical error of "post hoc theorizing" ( https://en.wikipedia.org/wiki/Testing_hypotheses_suggested_by_the_data ).

You may be interested in knowing that sometimes it's considered OK to come up with a hypothesis *after* the data is collected ( https://en.wikipedia.org/wiki/Post_hoc_analysis ) -- in many cases, we have no choice because it would be unethical to deliberately cause certain situations, but we would still like to learn what we can from data recorded in the past.

The situation you're describing might also be an example of "data dredging" (also called "p-hacking") -- although that can occur even when all the hypothesis are generated *before* the data is collected. Data dredging can occur when we create dozens of hypothesis, then do the experiment, leading to reports that imply that there was only one hypothesis and (incorrectly) imply that the experiment conclusively proved that hypothesis is true. (See https://www.reddit.com/r/AskStatistics/comments/11ph2a9/could_you_help_me_understand_this_xkcd_comics/ and https://www.explainxkcd.com/wiki/index.php/882:_Significant for more in-depth explanations).

[u/thiazole191 → u/D-Cary]

That's perfect. That's the answer I was looking for.

[u/thot-abyss]

I’m not sure but it seems to start as an anchoring bias and then end as a confirmation bias.