Edinburgh Research Explorer

Autoencoding Variational Inference for Topic Models

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Related Edinburgh Organisations

Open Access permissions

Open

Documents

Original languageEnglish
Title of host publicationProceedings for the 5th International Conference on Learning Representations
Number of pages12
Publication statusE-pub ahead of print - 26 Apr 2017
Event5th International Conference on Learning Representations - Palais des Congrès Neptune, Toulon, France
Duration: 24 Apr 201726 Apr 2017
https://iclr.cc/archive/www/2017.html

Conference

Conference5th International Conference on Learning Representations
Abbreviated titleICLR 2017
CountryFrance
CityToulon
Period24/04/1726/04/17
Internet address

Abstract

Topic models are one of the most popular methods for learning representations of text, but a major challenge is that any change to the topic model requires mathematically deriving a new inference algorithm. A promising approach to address this problem is autoencoding variational Bayes (AEVB), but it has proven difficult to apply to topic models in practice. We present what is to our knowledge the first effective AEVB based inference method for latent Dirichlet allocation (LDA), which we call Autoencoded Variational Inference For Topic Model (AVITM). This model tackles the problems caused for AEVB by the Dirichlet prior and by component collapsing. We find that AVITM matches traditional methods in accuracy with much better inference time. Indeed, because of the inference network, we find that it is unnecessary to pay the computational cost of running variational optimization on test data. Because AVITM is black box, it is readily applied to new topic models. As a dramatic illustration of this, we present a new topic model called ProdLDA, that replaces the mixture model in LDA with a product of experts. By changing only one line of code from LDA, we find that ProdLDA yields much more interpretable topics, even if LDA is trained via collapsed Gibbs sampling.

Event

5th International Conference on Learning Representations

24/04/1726/04/17

Toulon, France

Event: Conference

Download statistics

No data available

ID: 31213966