Neural Approximate Sufficient Statistics for Implicit Models

Yanzhi Chen; Dinghuai Zhang; Michael U. Gutmann; Aaron Courville; Zhanxing Zhu

Neural Approximate Sufficient Statistics for Implicit Models

Yanzhi Chen, Dinghuai Zhang, Michael U. Gutmann, Aaron Courville, Zhanxing Zhu

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract / Description of output

We consider the fundamental problem of how to automatically construct summary statistics for implicit generative models where the evaluation of likelihood function is intractable but sampling / simulating data from the model is possible. The idea is to frame the task of constructing sufficient statistics as learning mutual information maximizing representation of the data. This representation is computed by a deep neural network trained by a joint statistic-posterior learning strategy. We apply our approach to both traditional approximate Bayesian computation (ABC) and recent neural likelihood approaches, boosting their performance on a range of tasks.

Original language	English
Title of host publication	Ninth International Conference on Learning Representations (ICLR 2021)
Number of pages	14
Publication status	Published - 4 May 2021
Event	Ninth International Conference on Learning Representations 2021 - Virtual Conference Duration: 4 May 2021 → 7 May 2021 https://iclr.cc/Conferences/2021/Dates

Conference

Conference	Ninth International Conference on Learning Representations 2021
Abbreviated title	ICLR 2021
City	Virtual Conference
Period	4/05/21 → 7/05/21
Internet address	https://iclr.cc/Conferences/2021/Dates

Keywords / Materials (for Non-textual outputs)

likelihood-free inference
bayesian inference
mutual information
representation learning

Access to Document

https://openreview.net/forum?id=SRDuJssQudLicence: All Rights Reserved

Cite this

@inproceedings{e45260f0342b4073bcfceff7839f5e19,

title = "Neural Approximate Sufficient Statistics for Implicit Models",

abstract = "We consider the fundamental problem of how to automatically construct summary statistics for implicit generative models where the evaluation of likelihood function is intractable but sampling / simulating data from the model is possible. The idea is to frame the task of constructing sufficient statistics as learning mutual information maximizing representation of the data. This representation is computed by a deep neural network trained by a joint statistic-posterior learning strategy. We apply our approach to both traditional approximate Bayesian computation (ABC) and recent neural likelihood approaches, boosting their performance on a range of tasks.",

keywords = "likelihood-free inference, bayesian inference, mutual information, representation learning",

author = "Yanzhi Chen and Dinghuai Zhang and Gutmann, {Michael U.} and Aaron Courville and Zhanxing Zhu",

year = "2021",

month = may,

day = "4",

language = "English",

booktitle = "Ninth International Conference on Learning Representations (ICLR 2021)",

note = "Ninth International Conference on Learning Representations 2021, ICLR 2021 ; Conference date: 04-05-2021 Through 07-05-2021",

url = "https://iclr.cc/Conferences/2021/Dates",

}

TY - GEN

T1 - Neural Approximate Sufficient Statistics for Implicit Models

AU - Chen, Yanzhi

AU - Zhang, Dinghuai

AU - Gutmann, Michael U.

AU - Courville, Aaron

AU - Zhu, Zhanxing

PY - 2021/5/4

Y1 - 2021/5/4

N2 - We consider the fundamental problem of how to automatically construct summary statistics for implicit generative models where the evaluation of likelihood function is intractable but sampling / simulating data from the model is possible. The idea is to frame the task of constructing sufficient statistics as learning mutual information maximizing representation of the data. This representation is computed by a deep neural network trained by a joint statistic-posterior learning strategy. We apply our approach to both traditional approximate Bayesian computation (ABC) and recent neural likelihood approaches, boosting their performance on a range of tasks.

AB - We consider the fundamental problem of how to automatically construct summary statistics for implicit generative models where the evaluation of likelihood function is intractable but sampling / simulating data from the model is possible. The idea is to frame the task of constructing sufficient statistics as learning mutual information maximizing representation of the data. This representation is computed by a deep neural network trained by a joint statistic-posterior learning strategy. We apply our approach to both traditional approximate Bayesian computation (ABC) and recent neural likelihood approaches, boosting their performance on a range of tasks.

KW - likelihood-free inference

KW - bayesian inference

KW - mutual information

KW - representation learning

UR - https://arxiv.org/abs/2010.10079

M3 - Conference contribution

BT - Ninth International Conference on Learning Representations (ICLR 2021)

T2 - Ninth International Conference on Learning Representations 2021

Y2 - 4 May 2021 through 7 May 2021

ER -

Neural Approximate Sufficient Statistics for Implicit Models

Abstract / Description of output

Conference

Keywords / Materials (for Non-textual outputs)

Access to Document

Other files and links

Fingerprint

Cite this