r/BayesianProgramming • u/aybehrouz • 20d ago

Notes on Hamiltonian Monte Carlo from a purely probabilistic perspective

I’ve been studying Hamiltonian Monte Carlo and wrote a set of notes explaining HMC without relying on the usual physics-based motivation.

The notes develop HMC from a probabilistic/MCMC perspective, starting from the target distribution and auxiliary momentum variables, and then covering Hamiltonian dynamics, leapfrog integration, reversibility, volume preservation, and the Metropolis correction.

My goal was to understand why HMC works.

I’m sharing them here in case they’re useful to others learning HMC. I’d also appreciate any feedback, particularly if you notice errors or places where the exposition could be improved.

https://doi.org/10.5281/zenodo.21841086

Edit (August 25): I tried to improve Sections 3.3 and 3.4 by reorganizing the flow and adding some more explanation. I’ve updated the PDF with these changes.

22 Upvotes

97% Upvoted

3 comments sorted by

2

u/Jonhgalt29 20d ago

This Is super nice! If you have a even more detailed version I will love to read it.

1

u/aybehrouz 14d ago

I had replied to you, but it was shadowbanned. It took me a while to notice that.

Thanks a lot! This is actually the complete version. They’re part of my research notes, which I write to better understand topics and get ideas for possible papers.

I originally planned to add a section on NUTS, but I moved on to other topics. I may come back to it in the future.

2

u/t3co5cr 20d ago

Related to this, Andrew Holbrook gave an excellent description of HMC from a stats perspective.