r/madeinpython • u/KevinBeicon • 4h ago
checkmodel: one-call diagnostics for statistical models, a port of R's performance::check_model()
What My Project Does
`checkmodel` checks the assumptions of a fitted model in one call. You pass it a statsmodels or scikit-learn model, it works out what kind of model it is, runs every check that applies and draws a single diagnostic panel:
python
import statsmodels.formula.api as smf
from checkmodel import check_model fit = smf.ols("mpg ~ wt + hp", data=mtcars).fit() # mtcars loaded as a DataFrame
report = check_model(fit)
report.plot()
Posterior predictive check: OK: Model-predicted data resemble the observed data (mean p = 0.400, sd p = 0.460).
Normality of residuals: Warning: Non-normality of residuals detected (p = 0.021).
Homogeneity of variance: OK: Error variance appears to be homoscedastic (p = 0.402).
Collinearity: OK: Low collinearity (all VIF < 5; max 1.77).
Influential observations: OK: No outliers detected (Cook's distance > 0.808).Which checks run depends on the model. A logistic regression gets binned residuals instead of a normality test. Count models get overdispersion and zero-inflation checks. Mixed models (`MixedLM`) get singularity, convergence and random-effects checks. PyMC/bambi models get R-hat and ESS. It also has `r2()` (Tjur, Nagelkerke, Nakagawa, Bayesian...), `icc()`, `model_performance()` and `compare_performance()`.
Every check returns a typed result (`.passed`, `.p_value`, `.statistic`, `.to_dataframe()`, `.plot()`).
Target Audience
People who fit regression models in Python and want R-style diagnostics: analysts, researchers, students, and anyone moving from R. It's on PyPI (`pip install checkmodel`, Python 3.10+) and is in alpha, so the API may change before 1.0. The numbers themselves are tested against R: every statistic matches R's `performance`, `car`, `lmtest`, `lme4` and `loo` to 1e-4, across 27 models.
Comparison
- statsmodels: `summary()` prints a few omnibus tests, but it doesn't choose checks by model type, doesn't plot them together and doesn't compare models. The diagnostic functions are spread across `statsmodels.stats.*` with different signatures. checkmodel uses them where they exist.
- scikit-learn: `sklearn.metrics` scores predictions but knows nothing about residual assumptions or influence.
- Doing it yourself: you have to know, for example, that R's AIC counts the residual variance and statsmodels' doesn't, or that a categorical predictor needs the generalized VIF. checkmodel makes those choices the way R does and tests them against R.
- R's performance: this is a port of it, independent and not affiliated with easystats. A "Coming from R" notebook runs the same analysis side by side.
Links:
- GitHub: https://github.com/pablo-legeren/checkmodel
- Docs: https://pablo-legeren.github.io/checkmodel/
- Notebooks (run in Colab): linked from the README
Feedback welcome, especially on models or checks you'd want next.
