On Bayesian bounds

We show that several important Bayesian bounds studied in machine learning, both in the batch as well as the online setting, arise by an application of a simple compression lemma. In particular, we derive (i) PACBayesian bounds in the batch setting, (ii) Bayesian log-loss bounds and (iii) Bayesian bounded-loss bounds in the online setting using the compression lemma. Although every setting has different semantics for prior, posterior and loss, we show that the core bound argument is the same. The paper simplifies our understanding of several important and apparently disparate results, as well as brings to light a powerful tool for developing similar arguments for other methods.

• Type: Conference
• Year: 2006
• Where: ICML
• Authors: Arindam Banerjee
• Keywords: Bayesian Bounded-loss Bounds, Bayesian Log-loss Bounds, ICML 2006, Important Bayesian Bounds, Machine Learning