Semi-Separable Hamiltonian Monte Carlo for Inference in Bayesian Hierarchical Models

Yichuan Zhang, Charles Sutton

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Sampling from hierarchical Bayesian models is often difficult for MCMC methods, because of the strong correlations between the model parameters and the hyperparameters. Recent Riemannian manifold Hamiltonian Monte Carlo (RMHMC) methods have significant potential advantages in this setting, but are computationally expensive. We introduce a new RMHMC method, which we call semi-separable Hamiltonian Monte Carlo, which uses a specially designed mass matrix that allows the joint Hamiltonian over model parameters and hyperparameters to decompose into two simpler Hamiltonians. This structure is exploited by a new integrator which we call the alternating blockwise leapfrog algorithm. The resulting method can mix faster than simpler Gibbs sampling while being simpler and more efficient than previous instances of RMHMC.
Original languageEnglish
Title of host publicationAdvances in Neural Information Processing Systems 27
EditorsZ. Ghahramani, M. Welling, C. Cortes, N.D. Lawrence, K.Q. Weinberger
PublisherCurran Associates Inc
Number of pages9
Publication statusPublished - 2014


Dive into the research topics of 'Semi-Separable Hamiltonian Monte Carlo for Inference in Bayesian Hierarchical Models'. Together they form a unique fingerprint.

Cite this