Bregman divergence as general framework to estimate unnormalized statistical models

Michael Gutmann, Jun-ichiro Hirayama

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We show that the Bregman divergence provides a rich framework to estimate unnormalized statistical models for continuous or discrete random variables, that is, models which do not integrate or sum to one, respectively. We prove that recent estimation methods such as noise-contrastive estimation, ratio matching, and score matching belong to the proposed framework, and explain their interconnection based on supervised learning. Further, we discuss the role of boosting in unsupervised learning.
Original languageEnglish
Title of host publicationProceedings of Conference on Uncertainty in Artificial Intelligence (UAI 2011)
Number of pages8
Publication statusPublished - 2011

Fingerprint Dive into the research topics of 'Bregman divergence as general framework to estimate unnormalized statistical models'. Together they form a unique fingerprint.

Cite this