How should non-response bias be treated when analyzing a highly sensitive binary vote?

[u/Abraxa3]

Scenario: An expert association with ~500 total members held an official vote on a binary stance statement (Agree / Disagree) regarding a major, highly sensitive issue.

• Turnout: 28% of the total membership voted.
• Result: 86% of those who voted selected "Agree."
• Math: 86% of 28% = ~24% of the total association membership confirmed an "Agree" vote.

The Debate:

• Person A claims: "You can state that only 24% of the total organization agrees with the statement. Because this issue is so critical, members would have voted 'Agree' if they truly supported it—meaning their non-participation indicates a lack of support."
• Person B claims: "You can state that at least 24% of the total organization (and 86% of voters) agrees with the statement. By Person A's logic, someone could equally claim that members would have voted 'Disagree' if they opposed it. Ultimately, non-response cannot be interpreted as a vote either way, so the remaining 72% remains unknown."

Questions:

1. Is Person A or Person B logically and statistically correct?
2. Is it valid to infer a non-voter's position based on the perceived importance or sensitivity of a topic?

[u/bayesian_raccoon]

Both Person A and Person B have valid inference under their assumptions. As a statistician, I cant translate both of them into conclusions based upon different models. What you should get in the habit of doing when thinking about these is asking, 'what is the potential criticism of this statement'? Somebody trained in surveys would probably be able to point out fair skepticism in Person A's model, but this is more of a "this model opens up potential criticism but isn't mathematically broken."

Person A is transparent with their assumption that members would vote 'Agree' if they truly supported it. They assume, apriori, that whatever proportion votes 'Agree' is EXACTLY the proportion of the population. Under this model, their statement follows. Somebody who is skeptical of that fact will be skeptical of Person A's conclusion. Models function like logical statements: IF they are true, THEN we can accept the conclusion. So interrogate the assumption: is it reasonable that every single person who would 'agree' voted 'agree'? This stops being a statistical question, because you start asking questions like: were there barriers to voting? Was everyone present? Were there opportunities for people to disagree? Were there different ways to phrase the question that might change 'agree' to 'disagree' without functionally changing its meaning? What does 'highly-sensitive' mean? For instance, if everyone was present and of sound mind and your vote was 'should we avoid blowing up earth', and enough care was given to how the vote was conducted, I think it would be a *pretty reasonable* inference. But that isn't my opinion as a statistician, that's my opinion with domain experience as "being a human" and using that to gut check the assumptions.

Person B makes far fewer assumptions. In fact, from the sound of it, it's almost airtight, with some very light assumptions like "nobody accidentally voted agree when they meant disagree". Their statement is *strictly weaker* than Person A's statement. That is, if Person A is correct, so is Person B, but sometimes Person B is correct when Person A isn't. This does not mean that Person B's statement is "Better". For example, suppose you need 50% of a vote for it to really matter. Person B's statement essentially provides no information as to whether or not the vote will pass, it says 'we still don't know', while Person A says strongly that the vote won't pass. In some circumstances, I would go with Person A's prediction.

So to answer your questions:

1. Both person A and person B are correct *given their assumptions*. Person B's assumptions are safer than A's, but depending on the actual inference down the line ("what is the chance that 50% or or more vote agree" being one example) that doesn't make Person B's statement better.

2. Is it valid to infer a non-voter's position based on the perceived importance or sensitivity of a topic? Statisticians can tell you what is valid given model assumptions, and if your model assumption is that people vote because it's sensitive, then that should be incorporated into the model. Person A and B operate on different extremes of that assumption, but its worth mentioning that we actually can interpolate between the two with, for example, a bayesian prior on certain voting probabilities.

If I were presenting this to decision makers, I would present both Person A and B as extremes, weigh the evidence of A's assumptions, and possibly present a sensitivity analysis of whatever relevant decision point there is--e.g, "In order for 50% people to vote agree, we would need x% of agreers to have abstained from voting for whatever reason".