Edinburgh Research Explorer

MADE: Masked Autoencoder for Distribution Estimation

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Related Edinburgh Organisations

Open Access permissions


Original languageEnglish
Title of host publicationProceedings of The 32nd International Conference on Machine Learning
Place of PublicationLille, France
PublisherJournal of Machine Learning Research: Workshop and Conference Proceedings
Number of pages9
Publication statusPublished - 2015
Event32nd international conference on machine learning - Lille, France
Duration: 6 Jul 201511 Jul 2015


Conference32nd international conference on machine learning
Abbreviated titleICML 2015
Internet address


There has been a lot of recent interest in designing neural network models to estimate a distribution from a set of examples. We introduce a simple modification for autoencoder neural networks that yields powerful generative models. Our method masks the autoencoder’s parameters to respect autoregressive constraints: each input is reconstructed only from previous inputs in a given ordering. Constrained this way, the autoencoder outputs can be interpreted as a set of conditional probabilities, and their product, the full joint probability. We can also train a single network that can decompose the joint probability in multiple different orderings. Our simple framework can be applied to multiple architectures, including deep ones. Vectorized implementations, such as on GPUs, are simple and fast. Experiments demonstrate that this approach is competitive with state-of-the-art tractable distribution estimators. At test time, the method is significantly faster and scales better than other autoregressive estimators.


32nd international conference on machine learning


Lille, France

Event: Conference

Download statistics

No data available

ID: 19848824