Faithful Inversion of Generative Models for Effective Amortized Inference

Stefan Webb, Adam Golinski, Rob Zinkov, N. Siddharth, Tom Rainforth, Yee Whye Teh, Frank Wood

Research output: Chapter in Book/Report/Conference proceedingChapter


Inference amortization methods share information across multiple posterior-inference problems, allowing each to be carried out more efficiently. Generally, they require the inversion of the dependency structure in the generative model, as the modeller must learn a mapping from observations to distributions approximating the posterior. Previous approaches have involved inverting the dependency structure in a heuristic way that fails to capture these dependencies correctly, thereby limiting the achievable accuracy of the resulting approximations. We introduce an algorithm for faithfully, and minimally, inverting the graphical model structure of any generative model. Such inverses have two crucial properties: (a) they do not encode any independence assertions that are absent from the model and; (b) they are local maxima for the number of true independencies encoded. We prove the correctness of our approach and empirically show that the resulting minimally faithful inverses lead to better inference amortization than existing heuristic approaches.
Original languageEnglish
Title of host publicationAdvances in Neural Information Processing Systems 31
EditorsS. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, R. Garnett
PublisherCurran Associates Inc
Number of pages11
Publication statusPublished - 8 Dec 2018
EventThirty-second Conference on Neural Information Processing Systems - Montreal, Canada
Duration: 3 Dec 20188 Dec 2018


ConferenceThirty-second Conference on Neural Information Processing Systems
Abbreviated titleNIPS 2018
Internet address


Dive into the research topics of 'Faithful Inversion of Generative Models for Effective Amortized Inference'. Together they form a unique fingerprint.

Cite this