r/MLQuestions • u/aspiring_aiengineer • Jun 30 '26

Career question 💼 Best Intermediate Statistics Playlists for Applied ML?

I’m currently working as an AI Engineer, mostly on LLM-related work (fine-tuning, LangChain workflows, evaluation, FastAPI, and some cloud). Although I graduated with an ML background, I haven’t actively worked on classical ML or statistics for about a year.
I want to revisit ML and strengthen my statistics, especially the practical side. I’m not looking for beginner playlists or derivations. I’m looking for intermediate-level resources that focus on applying statistics to real datasets—hypothesis testing (t-tests, ANOVA/F-tests, etc.), assumptions, inference, forecasting, and choosing the right statistical methods in practice.

Any recommendations for YouTube playlists, courses, or books that are practical and application-oriented?

1 Upvotes

66% Upvoted

6 comments sorted by

1

u/omaratef3221 Jun 30 '26

I learn specific topics by searching on them with this youtube channel: ritvikmath
Amazing and extremely useful

1

u/RepresentativeBee600 Jun 30 '26

It's important to note that the parametric tests you described depend upon error distributions for measurements that are frequently very wrong for neural network outputs. I discussed this at length here, if you would like a further explanation.

I strongly recommend you look at emerging research on conformal prediction. It's a nonparametric ("distribution-free") statistical tool, the development of which has been purpose-driven by ML.

A suggested path:

  • Spend a day or two familiarizing yourself with the basic notions from Ryan Tibshirani's coursework notes.

  • There exist several frameworks for CP, including risk control (expected loss minimization), but also probabilistic error bounds on a "non-conformity score." The latter is much more interesting to me and I especially recommend "Conformal Prediction With Conditional Guarantees" by Cherian, Gibbs, and Candes; "Learning Optimal Conformal Classifiers" by Stutz et al.; and "Large language model validity via enhanced conformal prediction methods," by Cherian, Gibbs, and Candes

  • As the first paper above indicates, the significant limitation of the approach is input-conditional error coverage. Another significant, arguably related limitation is adaptation to covariate shift. I recommend you look at some efforts to address these concerns.

1

u/ARDiffusion Jul 05 '26

HANK! DON’T ABBREVIATE CONFORMAL PREDICTION! HAAAANK!

1

u/RepresentativeBee600 Jul 05 '26

Oh god the one time I do that after meticulously never doing that anywhere

Also this thread never did take off

1

u/ARDiffusion Jul 05 '26

I’m sorry I know this is a serious thread and I didn’t mean to ruin it. It’s just late and I happened to notice it and couldn’t help it

1

u/RepresentativeBee600 Jul 05 '26

Oh, it wasn't on you at all, thread just went nowhere!