Hamiltonian Monte Carlo

Hamiltonian Monte Carlo (HMC) is a type of Markov Chain Monte Carlo (MCMC) method for efficiently sampling from probability distributions. It was developed by Radford M. Neal in the 1990s. HMC works by introducing extra momentum variables into the sampling process, which allows for more efficient sampling along directions where the distribution has relatively large changes in value, as well as allowing for ‘bouncing’ off of regions where the distribution changes quickly. It is usually used in Bayesian inference, but can also be used in other types of learning or optimization tasks.

HMC is specifically useful in cases where the probability distribution is multi-modal (i.e., there is more than one distinct region of the distribution) or has strong curvature. It can also be useful in cases where the probability distribution changes quickly in some areas. HMC is particularly useful in cases where the probability distribution is a function of several continuous variables, since this can make it difficult to sample efficiently using MCMC methods such as Gibbs sampling.

In general, HMC is less efficient than other MCMC methods in terms of the number of samples that need to be taken in order to get an accurate estimate of the posterior probability distribution; however, it can often be faster in practice since it does not require taking many steps in order to sample from a particular region of the probability distribution.

The term “Hamiltonian” in Hamiltonian Monte Carlo comes from the fact that the dynamics of a Markov Chain can be expressed as a Hamiltonian system in a potentially high dimensional phase space. This allows for a more efficient representation of the samples compared to a regular MCMC representation, since the dynamics of the chain are expressed as a differential equation.