Evaluating probabilities under high-dimensional latent variable models

Iain Murray, Ruslan Salakhutdinov

Research output: Chapter in Book/Report/Conference proceedingConference contribution


We present a simple new Monte Carlo algorithm for evaluating probabilities of observations in complex latent variable models, such as Deep Belief Networks. While the method is based on Markov chains, estimates based on short runs are formally unbiased. In expectation, the log probability of a test set will be underestimated, and this could form the basis of a probabilistic bound. The method is much cheaper than gold-standard annealing-based methods and only slightly more expensive than the cheapest Monte Carlo methods. We give examples of the new method substantially improving simple variational bounds at modest extra cost.
Original languageEnglish
Title of host publicationAdvances in Neural Information Processing Systems 21
EditorsD. Koller, D. Schuurmans, Y. Bengio, L. Bottou
Number of pages8
Publication statusPublished - 2009


Dive into the research topics of 'Evaluating probabilities under high-dimensional latent variable models'. Together they form a unique fingerprint.

Cite this