The Bayesian backfitting relevance vector machine

Traditional non-parametric statistical learning techniques are often computationally attractive, but lack the same generalization and model selection abilities as state-of-the-art Bayesian algorithms which, however, are usually computationally prohibitive. This paper makes several important contributions that allow Bayesian learning to scale to more complex, real-world learning scenarios. Firstly, we show that backfitting -- a traditional non-parametric, yet highly efficient regression tool -- can be derived in a novel formulation within an expectation maximization (EM) framework and thus can finally be given a probabilistic interpretation. Secondly, we show that the general framework of sparse Bayesian learning and in particular the relevance vector machine (RVM), can be derived as a highly efficient algorithm using a Bayesian version of backfitting at its core. As we demonstrate on several regression and classification benchmarks, Bayesian backfitting offers a compelling alternative...

• Type: Conference
• Year: 2004
• Where: ICML
• Authors: Aaron D'Souza, Sethu Vijayakumar, Stefan Schaal
• Keywords: ICML 2004, Machine Learning, Non-parametric Statistical Learning, Sparse Bayesian Learning, State-of-the-art Bayesian Algorithms