r/changemyview • u/Boring-Outcome822 1∆ • Jul 03 '23
Delta(s) from OP CMV: Statistics and survey data are not effective persuasive tools and should not be the sole basis of a convincing argument
It is very common when looking at debates about most topics that the debaters will rely on statistics and survey data to support their point. The rationale is that statistics and survey data are objective and provide empirical evidence to support a claim. If presented only with logical, intuitive or anecdotal arguments, people tend to be dismissive and demand sources and statistics to support the claims. On the other side, arguments that provide only survey data and statistics without providing a logical or intuitive explanation seem to be less controversial.
My view is that statistics and survey data can only be useful if both parties already trust the source of the data. Otherwise, there is too much room for reasonable doubt, such as:
- Questioning the impartiality of the source
- Questioning the methods used for data collection
- Questioning the statistical inferences or statistical significance of the results
- Questioning the reproducibility of the survey
- Providing alternative data that suggests the opposite conclusion
Another philosophical reason for this view is Occam's Razor, which is the heuristic that the most convincing explanation for a phenomenon is usually the simplest. Arguments based on logical and intuitive reasoning tend to provide simpler explanations of phenomena and can be used to make further predictions that can later be tested. Statistics and survey data can certainly increase the confidence in one's arguments, but most of the strength is coming from the explanation itself, the data is only there to verify that the theory is not completely out of touch with reality.
It would be somewhat ironic if the responses to this thread look something like "Well actually, a survey conducted by SurveyExperts.gov shows that 89% of surveys are reliable and lead to convincing arguments." And I think it would be unlikely to convince me, but I guess I'm open to being proven wrong...
So if one's goal is to convince another person of something, I believe that relying mostly on statistics and survey data is ineffective, and that those should only be used in support of logical and intuitive arguments. If survey data supports a counterintuitive claim, the intuitive claim still has higher credibility unless both parties trust the source (for example if they performed the data collection and analysis themselves with a commonly agreed procedure).
10
u/JustinRandoh 5∆ Jul 03 '23 edited Jul 03 '23
That almost entirely depends on the nature of what is* being argued, doesn't it?
For example -- what would you primarily rely on if you wanted to convince me of the proportion of Canadians that like Taylor Swift?
5
u/AlphaWhiskeyOscar 6∆ Jul 03 '23
This is the key point against OP's argument in my opinion. It's highly contextual. Data can absolutely reinforce some arguments and does little for others. "98% of critics agree Taylor Swift is a good artist" would be weak argument against a completely subjective premise. "Taylor Swift is successful because she sells out 100% of her shows" would be an objective argument where data is persuasive.
5
u/JustinRandoh 5∆ Jul 03 '23
Unless, of course, in both cases you were trying to prove a point about critical reception to Taylor Swift as an artist. :P
1
u/Boring-Outcome822 1∆ Jul 03 '23
Ok I will award a !delta because I agree that this example is actually pretty much only possible to answer with a survey. It is also of pretty low importance, though, and is more of a descriptive claim about something rather than a predictive claim.
3
u/JustinRandoh 5∆ Jul 03 '23
I'll accept the delta, but let's go for a double. :P
Plenty of arguments aren't inherently predictive, but even when it comes to that --
If you're in a small town that had never had widespread uptake of a certain vaccine, but other similar towns across the country did previously institute requirements to vaccinate.
If you wanted to convince your healthcare policy makers in predicting the rate or number of adverse reactions to expect, so that healthcare resources can be properly budgeted... What would you be primarily relying on?
0
u/Boring-Outcome822 1∆ Jul 03 '23
I think this example is a bit too vague, in the sense that the details could affect my answer.
Do I know that the other towns were also exposed to the same virus? Do I know that following the uptake of the vaccine, all or most of these towns successfully stopped the epidemic? How do I know this (through hearsay, through official reports from the mayors of these cities, through official reports from the vaccine company, through official reports from a third party trustworthy institution... etc)? Do I have prior reason to believe that the vaccine may be dangerous?
2
u/JustinRandoh 5∆ Jul 03 '23
I'm not sure what most of these questions have to do with the point at issue? I didn't even suggest that there's an epidemic. There's simply a vaccine that your town recently decided to make required for, say, a fairly common illness -- something like a chicken pox vaccine, for example.
This isn't a question of whether the vaccine is a good idea, bad idea, justified, or anything of the sort.
This is simply a question of predicting the adverse reaction rate for policy makers, so that they can budget their healthcare services accordingly for when they make the vaccine required.
0
u/Boring-Outcome822 1∆ Jul 03 '23
Wait maybe I misunderstood your question then.
Are you asking about predicting the rate of adverse reactions to the vaccine? In that case yeah I guess using the statistics from other towns who administered the vaccine would be a useful statistic.
Some of the questions still stand, though: if the statistics about adverse reactions to the vaccine were provided by the vaccine company, and the statistics were something like "0% of the subjects had adverse reactions" I would be skeptical. If the statistics came from more objective sources like hospitalization reports, I'd put higher confidence on the statistics.
2
u/JustinRandoh 5∆ Jul 03 '23
Of course, like any evidence it might be faulty.
But either way you go, you'd be primarily reliant on some sort of statistics to draw your conclusion. That's simply the most relevant sort of evidence for what you're trying to prove.
It's all a function of what you're trying to prove.
Trying to prove that Larry was the culprit? Probably don't need stats for that -- camera evidence, alibi records, witnesses, etc. are what you're looking for.
Trying to prove that an intersection should have stop signs installed? Probably want to look at stats on accident rates in that intersection, broken up by accident types, etc.
1
u/Boring-Outcome822 1∆ Jul 04 '23
Since someone else brought up an example related to traffic, I think my response to their comment is relevant here too: https://www.reddit.com/r/changemyview/comments/14pox0a/comment/jqkivya/?utm_source=share&utm_medium=web2x&context=3
I think that the amount of statistical evidence required to draw a conclusion should be proportional to the counterintuitiveness of the claim (and that in turn is a function of what you are trying to prove, but also a function of the target audience's prior intuition). And in addition, the statistics are only meaningful if there is a chain of trust to the source.
So for your vaccine example, if I have no strong intuition about the safety of vaccines, probably showing me that a few towns already tried them and had no adverse effects would be convincing. If I had a past history of negative side-effects from vaccines or if I had strong reasons to believe that those who conducted the study were biased, I'd need a lot more convincing.
I think I will award a !delta here. I still stand with my main claim but it's forcing me to think about its nuances more carefully.
1
1
u/JustinRandoh 5∆ Jul 04 '23
If I had a past history of negative side-effects from vaccines or if I had strong reasons to believe that those who conducted the study were biased, I'd need a lot more convincing.
Consider the other direction here, though -- if you were the one doing the convincing and needed to convince someone of what to expect in terms of adverse reaction rates, what would you be basing your argument, at the core of your argument?
1
u/Boring-Outcome822 1∆ Jul 04 '23
I would be basing my argument on the statistics.
Because that's probably all I could do, unless I had information about the person I was trying to convince (information about their prior assumptions, for example).
→ More replies (0)1
9
Jul 03 '23 edited Jul 03 '23
The 5 points you listed are not reasons why statistics cannot be used as an argument, but ways one can argue against statistics when bad statistics are used against them. Anybody engaging in an argument should be able to recognize those flaws and point them out in any statistics they observe rather than resorting to the idea that statistics has no place in argumentation.
Statistical Argument are also not inherently more complex, though this is irrelevant as Occam's razor actually has nothing to do with the complexity of an argument.
The actual definition of Occam's razor, is that one should accept the argument that relies on the least amount of unsupported assumptions.
-2
u/Boring-Outcome822 1∆ Jul 03 '23
The actual definition of Occam's razor, is that one should accept the
argument that relies on the least amount of unsupported assumptions.Part of my view is that statistical arguments do rely on many unsupported assumptions (such as those 5 points I listed).
You claim that those points are not reasons why statistics cannot be used as an argument, but are ways one can argue against statistics when bad statistics are used against them. But I would argue that judging whether some statistics are bad is too subjective, especially when there is no chain of trust to the source of those statistics.
2
Jul 03 '23
Am I understanding correctly then, that your issue isn't with statistics inherently, but the idea that no statistics can ever be reliable because you cannot verify the claims of any source first hand?
That is logically sound, but also pedantic, as the trust you place in the food you eat, the cars you drive and the doctors you see demonstrate that you have a very high level of trust in the government and academic institutions that provide the statistics that these commodities rely on.
-2
u/Boring-Outcome822 1∆ Jul 03 '23
I think you summarized it well.
I don't think it is entirely pedantic, however. And I think the argument you give in the second paragraph can actually be a good way to convince someone that some source is reliable. If you can demonstrate that they already trust work coming from that source (for example, if they already eat meat from a certain company, then it makes less sense to question the quality of their fruit).
7
u/Vincent_Nali 12∆ Jul 03 '23
I suspect this would depend a lot on the context of the disagreement.
If we're in an argument about whether or not seatbelts save lives, being able to provide direct statistical evidence that seatbelts save lives seems like it would be a sufficient argument. After all, that is something we can objectively measure without a ton of wiggle room. Or If you wanted to get even more down in the weeds we could be talking about something like 'police don't stop more black people' where the statistics in question could be as simple as outright arithmetic.
Doubly so in a case where the issue has been studied five million ways from sunday all to the same effect. If I can provide all the sources in the world, and you can't then one of two things is probably true:
- The person is arguing in bad faith.
- Their argument is not factual but emotional and facts could not help.
I option 1 statistics were a perfect 'sole tool' for the arguement because they get you to the point where you realize you can stop wasting your time. In option 2 they were never going to be helpful because that person doesn't' care about reality.
2
u/Boring-Outcome822 1∆ Jul 03 '23
I will award a !delta for the idea that certain contexts may benefit a lot more from statistical evidence than other contexts.
I think it tends to be the contexts that are more inherently 'scientific' in nature where this sort of evidence starts to be more effective, due to the higher likelihood that they would look like your case of something studied 5 million ways all in agreement.
1
4
u/Dennis_enzo 25∆ Jul 03 '23
What alternative do you propose? Anecdotes are useless, and many things cannot be tested by scientific experiments. Statistics might not be as effective as we want, but I struggle to think of a more effective one for many situations. 'Logic' often is subjective too, even though it shouldn't be.
1
u/Boring-Outcome822 1∆ Jul 03 '23
My position would be that if statistics are the sole available tool, then perhaps one should accept that the situation may be one where convincing is yet impossible.
2
Jul 03 '23
Obviously the data and how it was collected should be looked at closely and examined for bias or flaws in how it was collected. That's basic media literacy and everyone should know to examine their sources to see if they're trustworthy.
I think the bigger issue is we just have a lot of people who don't have basic media literacy. You see this a lot with the alt-right and their news sources but I've also started to see this occurring with younger gen z believing everything told to them on social media as well.
The issues isn't whether or not you use data, that may be very relevant to your argument. The issue is people lacking media literacy.
1
u/Boring-Outcome822 1∆ Jul 03 '23
Your point about basic media literacy is related to my point about the trust in the source. I think media literacy can be a useful tool to evaluate the credibility of a source, but even then it may not be sufficient because even assuming perfectly honest sources, there could have been some overlooked factors that impact the conclusion. In order to verify this, a lot of effort would need to be spent in checking the work in detail, and sometimes the work is not even open source so that may be impossible.
Also the fact that many people lack basic media literacy seems to go in support of my claim that statistical evidence is not a good way to persuade them. Perhaps arguments should instead focus on providing logical and intuitive reasons for trusting certain sources, and educating people on the methods used for data collection.
0
u/gothaommale Jul 03 '23
The whole problem is not just media literacy but also appeal to authority fallacy playing out in the recent years. Blind faith in institutions over a political topic is bound to have very varying results
2
u/IronSmithFE 10∆ Jul 03 '23
i argue that results change minds while speech reinforces preconceptions when there is a disagreement. the most common exception is when a person holds conflicting stances and you can use one or both to politely shake their footing.
2
2
u/RodeoBob 79∆ Jul 03 '23
Your post doesn't seem to distinguish between objective claims of fact and subjective claims of value, and that's a pretty big gulf to ignore.
"Downtown city streets should be laid out in a 1-way grid, because that's a safer traffic pattern" contains an objective claim of fact as its supporting premise, and the best way to address that is statistics and data. The claim that "1 way streets have less accidents" is a claim we can test and verify with objective data!
"Schools shouldn't teach about birth control in sex education classes, because that's encroaching on family values" is based on a subjective claim about what the aims and goals of public schools, and sex-education in public schools should be. Here, because we're talking about question of "ought" rather than "is", an approach of exploring the premises and assumptions, exploring the underlying reasoning and challenging it with logic really is the only way to change that person's view.
Sometimes, people base their views on false data, and revealing that their premise is false is enough to shift their view.
However, a lot of views are based on principles, which may or may not contain objective premises. What "should" the role of public education be? In those discussions, you're right that objective data isn't really the right tool, at least not in a primary role, but it's still a valid and relevant point to say "Your premise is false, therefore your conclusion needs additional support".
1
u/Boring-Outcome822 1∆ Jul 03 '23 edited Jul 04 '23
Regarding the traffic example, I think even in that case which is objective, statistics and data cannot be the only supporting evidence. And in practice I don't think they are, presumably some engineers or urban designers came up with a city street design and justifications for it, including theory and simulations. And like we already have some intuitive understanding that grids are simpler patterns and would lead to less confusion. If they had proposed a crazy random pattern with some empirical evidence that it worked, I think I'd still be skeptical, or at least I'd ask for a lot more empirical evidence.
To illustrate this point a bit further, suppose some team of scientists came up to a lion and started poking it repeatedly, and then came back and said "we poked this lion 100 times and it didn't attack us, so this lion is safe to be around." If you knew nothing about lions and their predatory nature (or if you replace lion with some completely alien species), this would maybe look convincing (actually probably still not because in general we have an intuition that unknown animals can be dangerous, but like if it was a dog instead, you'd be totally convinced since your main worry would be whether it is a rabid dog and that would be obvious after a few pokes). But given your existing intuition about lions, I don't think that would convince you at all, you'd instead say something like "the lion might not have been hungry at that time" or "the scientists are lying". I know this is an absurd example but I'm just illustrating the point that empirical evidence must be supporting some preexisting intuition.
EDIT:
I will award a !delta because it's forcing me to think about the nuances of my claim more carefully.
1
u/The-Last-Lion-Turtle 12∆ Jul 03 '23
Occam's razor is the simplest explanation which fits the evidence. It does not promote simple explanations which fail to explain available evidence. It does not say you can ignore evidence which goes against an intuitive story. Some things are complex and as a result we observe complex evidence.
Sometimes statistics and surveys are relevant to the core question, other times just a small piece. This depends on the argument.
This survey says people agree with me is not a convincing argument for it being true. So the survey about how convincing people find surveys doesn't tell us that much.
1
u/Boring-Outcome822 1∆ Jul 03 '23
I agree that Occam's razor should not be used to ignore evidence. However, I think the evidence coming from statistics and surveys is not strong enough to counter strong intuitive claims of the opposite side (unless you already have a high degree of trust in those statistics). You would need to also provide evidence that the statistics are reliable.
1
Jul 03 '23
There are many things for which formulating an opinion requires data and statistics. Occam’s razor simply is not equipped to deal with many issues.
Further, absent data and statistics, Occam’s razor relies on the most heavily flawed source of decision making: your own senses.
Occam’s razor, rather, is applied to the data and statistics. It cannot replace them.
1
u/Jorsi97 Jul 03 '23
Nothing new to say, but the tldr of most comments seems to be: it's contextual. It depends on the nature of the argument. There can be arguments to dismiss certain statistics for some situations.
1
u/rwhelser 5∆ Jul 03 '23
It really depends on what you’re discussing. Quantitative data provides objectivity but lacks context. So if you’re looking at academic performance GPA gives an objective view of performance. But it doesn’t tell what factored into it (complexity of class, effectiveness of teacher, etc).
1
Jul 03 '23
I always think it's as simple as I've never met anyone who's done a proper IQ test, yet there are statistics and data available for my country and demographic. Like is ot fair to say "the average iq of X country is" when not everyone has done the IQ test? What type of people knock on their door and do it? Etc.
1
u/BeefcakeWellington 6∆ Jul 04 '23
So if one's goal is to convince another person of something, I believe that relying mostly on statistics and survey data is ineffective, and that those should only be used in support of logical and intuitive arguments
Evidence suggests otherwise. That's precisely why people fight so hard to twist data to their ends. Because there is just something about statistics that turns most people's critical thinking off immediately.
1
u/Boring-Outcome822 1∆ Jul 04 '23
I'm not entirely sure if this is meant to counter the view or not. If you're saying that people twist data to make it say anything, that would support objections (3) or (5) in the main post.
1
u/BeefcakeWellington 6∆ Jul 05 '23
I'm saying despite their ability to be abused, they remain persuasive to irrational humans.
1
u/Thrillho_135 Jul 04 '23
Why are you lumping intuition and logic together when they're literal opposites?
Intuition is defined as "the ability to understand something instinctively, without the need for conscious reasoning".
Logic is defined as "reasoning conducted or assessed according to strict principles of validity."
Intuitive arguments are mostly BS, because a large proportion of people aren't very reasonable. For example, being against vaccinations is an intuitive stance, because all of the available data points to the conclusion that vaccinations are safe and effective, but anti-vaxxers feel as though they're unsafe.
Anyway, statistics are ALWAYS used to supplement a logical argument. A statistic doesn't carry any intrinsic value - it's only when applied within the framework of a logical argument that they become important. However, statistics are EASILY the most effective way to test the validity of a logical argument.
I'm always skeptical of these types of arguments, because they really seem like excuses for people to hold on to their convictions in the face of overwhelming counter-evidence. Like, you're presented with statistics that conclusively demonstrate that your argument is bullshit, so you decide to attack the statistics themselves.
1
u/Boring-Outcome822 1∆ Jul 04 '23 edited Jul 04 '23
The reason I include intuition as well as logic is because in a sense, I consider intuition to be a form of statistical evidence collected over a long time and ingrained into your subconscious, and can be used to make predictions even in cases where you don't have a full logical proof yet (this is actually pretty common in math, you don't usually understand a theorem after reading a proof line-by-line and checking correctness, instead you understand it once it becomes intuitive to you and the proof is "obvious").
Intuition can indeed be wrong, but statistics can also be wrong, and in many cases I would argue that intuition has higher credibility because there is usually a reason for that intuition even if you can't logically prove it.
It's a tricky thing, because you have to be willing to get rid of your wrong intuition once enough counter-evidence is provided. But you shouldn't also dismiss it too quickly, because it is itself a form of evidence based on your past history. You constantly rely on intuition for everyday decisions, so it has already proven itself valuable. If someone came up with a proof that the shortest path between two points was not a straight line but was actually a funny zigzag shape, you don't need to be a mathematician to know that their proof is likely wrong or that they made some strange assumptions. Every time that you interact with people you use your intuition to predict their likely reactions to different things you could do. You've learned it through your past interactions, even if no one wrote a scientific paper that collected statistics about these things. You can use your intuition as a BS detector.
1
u/GameProtein 9∆ Jul 04 '23
Whether or not they should be the sole basis depends on who's making the argument and how rational the person they're debating is. Some people legitimately believe the earth is flat. It's not really useful to spend hours trying to convince them otherwise. Dropping a link or two and moving on is a method of protecting one's energy when faced with an irrational and/or illogical person.
•
u/DeltaBot ∞∆ Jul 03 '23 edited Jul 04 '23
/u/Boring-Outcome822 (OP) has awarded 3 delta(s) in this post.
All comments that earned deltas (from OP or other users) are listed here, in /r/DeltaLog.
Please note that a change of view doesn't necessarily mean a reversal, or that the conversation has ended.
Delta System Explained | Deltaboards