Conditional sum-product networks: Modular probabilistic circuits via gate functions

Xiaoting Shao, Alejandro Molina, Antonio Vergari, Karl Stelzner, Robert Peharz, Thomas Liebig, Kristian Kersting

Research output: Contribution to journalArticlepeer-review

Abstract

While probabilistic graphical models are a central tool for reasoning under uncertainty in AI, they are in general not as expressive as deep neural models, and inference is notoriously hard and slow. In contrast, deep probabilistic models such as sum-product networks (SPNs) capture joint distributions and ensure tractable inference, but still lack the expressive power of intractable models based on deep neural networks. In this paper, we introduce conditional SPNs (CSPNs)—conditional density estimators for multivariate and potentially hybrid domains—and develop a structure-learning approach that derives both the structure and parameters of CSPNs from data. To harness the expressive power of deep neural networks (DNNs), we also show how to realize CSPNs by conditioning the parameters of vanilla SPNs on the input using DNNs as gate functions. In contrast to SPNs whose high-level structure can not be explicitly manipulated, CSPNs can naturally be used as tractable building blocks of deep probabilistic models whose modular structure maintains high-level interpretability. In experiments, we demonstrate that CSPNs are competitive with other probabilistic models and yield superior performance on structured prediction, conditional density estimation, auto-regressive image modeling, and multilabel image classification. In particular, we show that employing CSPNs as encoders and decoders within variational autoencoders can help to relax the commonly used mean field assumption and in turn improve performance.
Original languageEnglish
Pages (from-to)298-313
Number of pages16
JournalInternational Journal of Approximate Reasoning
Volume140
Early online date4 Nov 2021
DOIs
Publication statusPublished - 1 Jan 2022

Keywords

  • Conditional probabilistic modeling
  • Probabilistic circuits
  • Sum-product-networks
  • Tractable inference
  • Variational inference

Fingerprint

Dive into the research topics of 'Conditional sum-product networks: Modular probabilistic circuits via gate functions'. Together they form a unique fingerprint.

Cite this