Masked Autoregressive Flow for Density Estimation

George Papamakarios; Theo Pavlakou; Iain Murray

Masked Autoregressive Flow for Density Estimation

George Papamakarios, Theo Pavlakou, Iain Murray

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Autoregressive models are among the best performing neural density estimators. We describe an approach for increasing the flexibility of an autoregressive model, based on modelling the random numbers that the model uses internally when generating data. By constructing a stack of autoregressive models, each modelling the random numbers of the next model in the stack, we obtain a type of normalizing flow suitable for density estimation, which we call Masked Autoregressive Flow. This type of flow is closely related to Inverse Autoregressive Flow and is a generalization of Real NVP. Masked Autoregressive Flow achieves state-of-the-art performance in a range of general-purpose density estimation tasks.

Original language	English
Title of host publication	Advances in Neural Information Processing Systems 30
Editors	I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, R. Garnett
Place of Publication	Long Beach, United States
Publisher	Curran Associates Inc
Pages	2335-2344
Number of pages	10
Publication status	Published - 9 Dec 2017
Event	Thirty-first Annual Conference on Neural Information Processing Systems - Long Beach Convention Center, Long Beach, United States Duration: 4 Dec 2017 → 9 Dec 2017

Publication series

Name	Advances in Neural Information Processing Systems
Volume	30
ISSN (Electronic)	1049-5258

Conference

Conference	Thirty-first Annual Conference on Neural Information Processing Systems
Abbreviated title	NIPS
Country/Territory	United States
City	Long Beach
Period	4/12/17 → 9/12/17

Access to Document

Masked Autoregressiv_PAPAMAKRIOS_DOA04092017_VOR_All_RIGHTSFinal published version, 548 KBLicence: All Rights Reserved

Iain Murray
- i.murrayed.acuk
- School of Informatics - Personal Chair of Machine Learning and Inference
- Institute for Adaptive and Neural Computation
- Data Science and Artificial Intelligence
Person: Academic: Research Active

Cite this

Papamakarios, G., Pavlakou, T., & Murray, I. (2017). Masked Autoregressive Flow for Density Estimation. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, & R. Garnett (Eds.), Advances in Neural Information Processing Systems 30 (pp. 2335-2344). (Advances in Neural Information Processing Systems; Vol. 30). Curran Associates Inc. https://papers.nips.cc/paper/6828-masked-autoregressive-flow-for-density-estimation

Papamakarios, George ; Pavlakou, Theo ; Murray, Iain. / Masked Autoregressive Flow for Density Estimation. Advances in Neural Information Processing Systems 30. editor / I. Guyon ; U. V. Luxburg ; S. Bengio ; H. Wallach ; R. Fergus ; S. Vishwanathan ; R. Garnett. Long Beach, United States : Curran Associates Inc, 2017. pp. 2335-2344 (Advances in Neural Information Processing Systems).

@inproceedings{bcf26e3bc11d4b2887cae3910f9f6402,

title = "Masked Autoregressive Flow for Density Estimation",

abstract = "Autoregressive models are among the best performing neural density estimators. We describe an approach for increasing the flexibility of an autoregressive model, based on modelling the random numbers that the model uses internally when generating data. By constructing a stack of autoregressive models, each modelling the random numbers of the next model in the stack, we obtain a type of normalizing flow suitable for density estimation, which we call Masked Autoregressive Flow. This type of flow is closely related to Inverse Autoregressive Flow and is a generalization of Real NVP. Masked Autoregressive Flow achieves state-of-the-art performance in a range of general-purpose density estimation tasks.",

author = "George Papamakarios and Theo Pavlakou and Iain Murray",

year = "2017",

month = dec,

day = "9",

language = "English",

series = "Advances in Neural Information Processing Systems",

publisher = "Curran Associates Inc",

pages = "2335--2344",

editor = "I. Guyon and Luxburg, {U. V.} and S. Bengio and H. Wallach and R. Fergus and S. Vishwanathan and R. Garnett",

booktitle = "Advances in Neural Information Processing Systems 30",

note = "Thirty-first Annual Conference on Neural Information Processing Systems, NIPS ; Conference date: 04-12-2017 Through 09-12-2017",

}

Papamakarios, G, Pavlakou, T & Murray, I 2017, Masked Autoregressive Flow for Density Estimation. in I Guyon, UV Luxburg, S Bengio, H Wallach, R Fergus, S Vishwanathan & R Garnett (eds), Advances in Neural Information Processing Systems 30. Advances in Neural Information Processing Systems, vol. 30, Curran Associates Inc, Long Beach, United States, pp. 2335-2344, Thirty-first Annual Conference on Neural Information Processing Systems, Long Beach, United States, 4/12/17. <https://papers.nips.cc/paper/6828-masked-autoregressive-flow-for-density-estimation>

Masked Autoregressive Flow for Density Estimation. / Papamakarios, George; Pavlakou, Theo; Murray, Iain.

Advances in Neural Information Processing Systems 30. ed. / I. Guyon; U. V. Luxburg; S. Bengio; H. Wallach; R. Fergus; S. Vishwanathan; R. Garnett. Long Beach, United States : Curran Associates Inc, 2017. p. 2335-2344 (Advances in Neural Information Processing Systems; Vol. 30).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Masked Autoregressive Flow for Density Estimation

AU - Papamakarios, George

AU - Pavlakou, Theo

AU - Murray, Iain

PY - 2017/12/9

Y1 - 2017/12/9

N2 - Autoregressive models are among the best performing neural density estimators. We describe an approach for increasing the flexibility of an autoregressive model, based on modelling the random numbers that the model uses internally when generating data. By constructing a stack of autoregressive models, each modelling the random numbers of the next model in the stack, we obtain a type of normalizing flow suitable for density estimation, which we call Masked Autoregressive Flow. This type of flow is closely related to Inverse Autoregressive Flow and is a generalization of Real NVP. Masked Autoregressive Flow achieves state-of-the-art performance in a range of general-purpose density estimation tasks.

AB - Autoregressive models are among the best performing neural density estimators. We describe an approach for increasing the flexibility of an autoregressive model, based on modelling the random numbers that the model uses internally when generating data. By constructing a stack of autoregressive models, each modelling the random numbers of the next model in the stack, we obtain a type of normalizing flow suitable for density estimation, which we call Masked Autoregressive Flow. This type of flow is closely related to Inverse Autoregressive Flow and is a generalization of Real NVP. Masked Autoregressive Flow achieves state-of-the-art performance in a range of general-purpose density estimation tasks.

M3 - Conference contribution

T3 - Advances in Neural Information Processing Systems

SP - 2335

EP - 2344

BT - Advances in Neural Information Processing Systems 30

A2 - Guyon, I.

A2 - Luxburg, U. V.

A2 - Bengio, S.

A2 - Wallach, H.

A2 - Fergus, R.

A2 - Vishwanathan, S.

A2 - Garnett, R.

PB - Curran Associates Inc

CY - Long Beach, United States

T2 - Thirty-first Annual Conference on Neural Information Processing Systems

Y2 - 4 December 2017 through 9 December 2017

ER -

Masked Autoregressive Flow for Density Estimation

Abstract

Publication series

Conference

Access to Document

Fingerprint

Profiles

Iain Murray

Cite this