Measure Transformer Semantics for Bayesian Machine Learning

Abstract. The Bayesian approach to machine learning amounts to inferring posterior distributions of random variables from a probabilistic model of how the variables are related (that is, a prior distribution) and a set of observations of variables. There is a trend in machine learning towards expressing Bayesian models as probabilistic programs. As a foundation for this kind of programming, we propose a core functional calculus with primitives for sampling prior distributions and observing variables. We define combinators for measure transformers, based on theorems in measure theory, and use these to give a rigorous semantics to our core calculus. The original features of our semantics include its support for discrete, continuous, and hybrid measures, and, in particular, for observations of zero-probability events. We compile our core language to a small imperative language that has a straightforward semantics via factor graphs, data structures that enable many efficient inference al...

• Type: Journal
• Year: 2011
• Where: ESOP
• Authors: Johannes Borgström, Andrew D. Gordon, Michael Greenberg, James Margetson, Jurgen Van Gael
• Keywords: Approximate Inference, Core Calculus, ESOP 2011, Programming Languages, Rigorous Semantics