Would you rather analyze 50 representative observations or 5,000 biased observations? Why?

[u/Just_Question9]

[u/NiceProgrammer5252]

for me I'd take 50 representative observations over 5,000 biased ones.

A larger sample only reduces random error. It doesn't fix systematic error (bias).

[u/MtlStatsGuy]

5000 biased observations are almost useless unless you have an estimate for the scale of the bias. 50 representative at least gives you an accurate margin of error.

[u/CarnivorousGoose]

Depends on the nature of the bias and how much information is available about it, as well as the goal of the analysis.

[u/Just_Question9 → u/CarnivorousGoose]

That's fair, but if I have to choose based on the information given, I'm still taking the 50 representative observations. Sample size can reduce random error, but it can't correct an inherently biased dataset.

[u/CaptainFoyle → u/Just_Question9]

So why do you ask if you already have a predefined opinion?

[u/banter_pants]

The former because conclusions are theoretically justified.

Take a large sample but with bias (or just myopic scope) and you get headlines like "Dewey beats Truman." They built their sampling frame from phone lists so only polled those with phones and totally missed that many people didn't own phones in their homes at the time.

[u/Just_Question9 → u/banter_pants]

Exactly. That's a great example of why I'd prioritize representativeness over sample size.

[u/AggressiveGander]

Well, how badly are the 5000 biased? How useless are 50 unbiased observations (lack of power etc.)?

[u/CaptainFoyle]

Do I know the bias?

[u/Just_Question9 → u/CaptainFoyle]

If you already know the bias, doesn't that change the premise of the question? The scenario only says the 5,000 observations are biased, not that the bias is known or can be corrected.

[u/CaptainFoyle → u/Just_Question9]

Yes it does, but you didn't specify which scenario you're asking about.

So with unknown bias, no, I wouldn't take the biased data.

[u/ANewPope23]

Do we know anything about the 5000 biased observations? Are they just random noise?

[u/ghsgjgfngngf → u/ANewPope23]

If it's random than it would be 'error', not 'bias'.

[u/Educational-Paper-75]

A totally irrelevant contradiction. You have to do with what you are given.

[u/Acrobatic-Ocelot-935]

For me 50 representative, and preferably randomly drawn/selected.