Functional Programming for Modular Bayesian Inference

Adam Scibior, Ohad Kammar, Zoubin Ghahramani

Research output: Contribution to journalArticlepeer-review

Abstract

We present an architectural design of a library for Bayesian modelling and inference in modern functional programming languages. The novel aspect of our approach are modular implementations of existing state-of-the-art inference algorithms. Our design relies on three inherently functional features: higher-order functions, inductive data-types, and support for either type-classes or an expressive module system. We provide a performant Haskell implementation of this architecture, demonstrating that high-level and modular probabilistic programming can be added as a library in sufficiently expressive languages.

We review the core abstractions in this architecture: inference representations, inference transformations, and inference representation transformers. We then implement concrete instances of these abstractions, counterparts to particle filters and Metropolis-Hastings samplers, which form the basic building blocks of our library. By composing these building blocks we obtain state-of-the-art inference algorithms: Resample-Move Sequential Monte Carlo, Particle Marginal Metropolis-Hastings, and Sequential Monte Carlo Squared. We evaluate our implementation against existing probabilistic programming systems and find it is already competitively performant, although we conjecture that existing functional programming optimisation techniques could reduce the overhead associated with the abstractions we use. We show that our modular design enables deterministic testing of inherently stochastic Monte Carlo algorithms. Finally, we demonstrate using OCaml that an expressive module system can also implement our design.
Original languageEnglish
Article number83
Number of pages29
JournalProceedings of the ACM on Programming Languages
Volume2
Issue numberICFP
Early online date30 Jul 2018
DOIs
Publication statusPublished - 1 Sep 2018
Event23rd ACM SIGPLAN International Conference on Functional Programming - St. Louis, United States
Duration: 23 Sep 201829 Sep 2018
https://icfp18.sigplan.org/home

Keywords

  • Anglican
  • Bayesian inference
  • Markov Chain Monte Carlo
  • Monte Carlo samplers
  • Sequential Monte Carlo
  • WebPPL
  • functional programming
  • higher-order functions
  • inductive types
  • machine learning
  • module systems
  • monad transformers
  • monads
  • probabilistic programming
  • type-classes
  • software libraries and repositories

Fingerprint

Dive into the research topics of 'Functional Programming for Modular Bayesian Inference'. Together they form a unique fingerprint.

Cite this