A Family of Computationally Efficient and Simple Estimators for Unnormalized Statistical Models

M. Pihlaja, M. Gutmann, A. Hyvärinen

Research output: Chapter in Book/Report/Conference proceedingConference contribution


We introduce a new family of estimators for unnormalized statistical models. Our family of estimators is parameterized by two nonlinear functions and uses a single sample from an auxiliary distribution, generalizing Maximum Likelihood Monte Carlo estimation of Geyer and Thompson (1992). The family is such that we can estimate the partition function like any other parameter in the model. The estimation is done by optimizing an algebraically simple, well defined objective function, which allows for the use of dedicated optimization methods. We establish consistency of the estimator family and give an expression for the asymptotic covariance matrix, which enables us to further analyze the influence of the nonlinearities and the auxiliary density on estimation performance. Some estimators in our family are particularly stable for a wide range of auxiliary densities. Interestingly, a specific choice of the nonlinearity establishes a connection between density estimation and classification by nonlinear logistic regression. Finally, the optimal amount of auxiliary samples relative to the given amount of the data is considered from the perspective of computational efficiency.
Original languageEnglish
Title of host publicationProc. Conf. on Uncertainty in Artificial Intelligence (UAI)
Place of PublicationCorvallis, Oregon
PublisherAUAI Press
Number of pages8
Publication statusPublished - 2010


Dive into the research topics of 'A Family of Computationally Efficient and Simple Estimators for Unnormalized Statistical Models'. Together they form a unique fingerprint.

Cite this