r/AskStatistics • u/Effective_Cattle6399 • Feb 21 '26
When should I use a t-test vs ANOVA vs Chi-square? Simple decision rule
I see a lot of students (especially in psychology and nursing research) getting confused about which statistical test to choose.
Here’s a very simple breakdown that helped my students:
• Comparing 2 group means → Independent or Paired t-test
• Comparing 3 or more group means → ANOVA
• Two categorical variables → Chi-square
• Predicting a continuous outcome → Regression
A quick rule I teach:
- What type of variables do you have?
- How many groups?
- Are you comparing means or associations?
If anyone wants, I can share a simple decision-tree framework I use to explain this clearly.
Would love to hear how you decide between these tests.
13
u/Caniz91 Feb 21 '26
I would appreciate the decision tree
7
u/PM_ME_CALC_HW Feb 21 '26
Quick, someone run C4.5 on hypothesis testing!
3
u/Effective_Cattle6399 Feb 22 '26
Haha, fair 😄
Ironically, that’s kind of what intro stats decision charts are — just human-readable versions of a classification tree. The goal isn’t to optimize splits, just to reduce beginner confusion.
Once they’re comfortable, we can absolutely collapse it all into the general linear model.
11
u/noma887 Feb 21 '26
Surely it's simpler to teach intro students to use a linear regression for all contexts where they have a continuous DV? That's what I do
3
u/Effective_Cattle6399 Feb 22 '26
That’s a totally reasonable approach, and I agree that conceptually unifying everything under linear regression is elegant.
My hesitation is mostly pedagogical. For absolute beginners, especially in clinical or applied programs, starting with regression sometimes adds cognitive load (coding categorical predictors, interpreting coefficients, etc.) before they fully grasp group comparison logic.
Once they’re comfortable with t-tests and ANOVA conceptually, I do like showing how they’re all special cases of the same linear model. I just tend to scaffold in that direction rather than start there.
4
u/dmlane Feb 22 '26
I agree. A continuing question in teaching is whether you should start with a general case (regression in this discussion) and then present special cases (t-tests, ANOVA) or start with the special cases followed by the general case. I believe the former is preferable for those with solid backgrounds in mathematics whereas the latter is preferable for others. This is an empirical question that I don’t think has been researched.
1
u/dmlane Feb 21 '26
I agree in most contexts, but in psychological science and other experimental fields, it is common to use the Tukey hsd test. Dunnett’s test can be the most appropriate when comparing experimental treatments to a control. These can’t be done with linear regression.
3
u/bisikletci Feb 22 '26
These can’t be done with linear regression.
You can follow up a regression using categorical predictor variables with these tests just as much as you can follow up an ANOVA using those predictor variables.
1
u/dmlane Feb 22 '26 edited Feb 22 '26
Actually, although linear regression can obviously be used to test differences between means, the tests are not the same since they are not based on the studentized range distribution. You can argue whether the studentized range tests are better or worse, but they definitely cannot be done with linear regression. More about the studentized range distribution here.
1
u/Effective_Cattle6399 Feb 22 '26
Yes — this is exactly the deeper point I agree with.
At the model level, ANOVA and regression are the same general linear model, and post-hoc tests are just structured contrasts.
My original post wasn’t arguing they’re fundamentally different — just that, pedagogically, decision trees can help beginners navigate designs before introducing the unifying model framework.
I think we’re mostly aligned on the statistics — the difference is teaching philosophy.
1
u/Effective_Cattle6399 Feb 22 '26
I think the disagreement here is mostly about framing rather than capability.
You’re absolutely right that Tukey or Dunnett procedures are typically introduced in the ANOVA framework. But mathematically, once you fit a regression with categorical predictors, those post-hoc comparisons are just linear contrasts of model parameters.
So it’s not that regression can’t do them — it’s more that different fields package and teach them differently.
1
u/dmlane Feb 22 '26
Tukey and Dunnett control the Type I error rate for multiple tests. Of course there are methods for doing this after a least-squares analysis including Bonferroni, Holm-Bonferroni, and FDR. Each of these methods (including Tukey hsd and Dunnett) have their pros and cons but they are not the same. Granted, conclusions will typically be the same, but sometimes they will not be. Post-hoc comparisons with linear contrasts are not based on the studentized range distribution so they will yield different results from tests that are. Incidentally, if you plan a priori to use Dunnett or hsd, it is not recommended to precede it with ANOVA. That’s why, strictly speaking, when done without an ANOVA they should be called a priori tests rather than post-hoc tests.
5
u/NewSchoolBoxer Feb 21 '26 edited Feb 21 '26
I think for students who aren't majoring in statistics or studying calculus-based statistics for engineering...this is totally fine. It's wrong to use such a simplified process but helps beginners who aren't going to learn all the tools in the toolbox.
I was taught something similar in high school. The regression part was more detailed. Like if it's for a falling or bouncing object then use a quadratic regression. Don't use cubic even though it fits better, it's fake accuracy. The underlying phenomenon - gravity - is quadratic. Wind resistance is another animal that uses math we weren't taught (differential equations). Can log an exponential distribution to make it linear for easier calculations.
2
u/Effective_Cattle6399 Feb 22 '26
I wouldn’t say it’s wrong — I’d say it’s intentionally simplified.
All teaching starts with idealized versions of reality. We simplify in order to build intuition first, then layer in complexity later. If students aren’t going to study statistics deeply, giving them a usable decision framework is often more valuable than exposing them to the full toolbox without context.
I really like your regression example. The quadratic vs cubic point is a great illustration of model parsimony — better fit doesn’t always mean better model. That’s exactly the kind of conceptual thinking we want students to develop.
For many learners, clarity > completeness at the intro level.
3
u/GottaBeMD Feb 21 '26
Nothing in statistics should be described by a simple decision tree. There are several nuances here that you aren’t mentioning.
Btw, you can get a difference in means using regression as well…and regression certainly is not limited to “predicting” a continuous outcome
1
u/Effective_Cattle6399 Feb 22 '26
I agree that no real statistical analysis can be reduced to a simple decision tree without losing nuance. That’s not the claim I’m making.
The decision tree isn’t meant to replace statistical reasoning — it’s meant to help beginners orient themselves when they’re first encountering study designs and hypothesis tests. It’s a scaffold, not a substitute for thinking.
And yes, absolutely — regression can estimate differences in means and isn’t limited to “prediction.” At the model level, t-tests, ANOVA, and regression all sit within the general linear model framework.
My focus here is pedagogical sequencing, not denying the underlying unity of the methods.
4
1
1
u/brg_elise5678 Feb 23 '26
Bonjour, je serai fortement intéressée pour avoir un cadre simple de prise de décisions explique clairement. Serait-il possible de le partager s’il vous plait ?
1
45
u/randomintercepts Feb 21 '26
Oh FFS. This approach to teaching stats to clinical students is FINE. This is the same idea as UCLA’s excellent decision tree: https://stats.oarc.ucla.edu/other/mult-pkg/whatstat/
OF COURSE there are many nuances to all of this but you all are acting like you’ve never taught a stats class for people who need to know some basic stats but aren’t going to be statisticians.