The Bayesian backfitting relevance vector machine

A. D'Souza, S. Vijayakumar, S. Schaal

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Traditional non-parametric statistical learning techniques are often computationally attractive, but lack the same generalization and model selection abilities as state-of-the-art Bayesian algorithms which, however, are usually computationally prohibitive. This paper makes several important contributions that allow Bayesian learning to scale to more complex, real-world learning scenarios. Firstly, we show that backfitting --- a traditional non-parametric, yet highly efficient regression tool --- can be derived in a novel formulation within an expectation maximization (EM) framework and thus can finally be given a probabilistic interpretation. Secondly, we show that the general framework of sparse Bayesian learning and in particular the relevance vector machine (RVM), can be derived as a highly efficient algorithm using a Bayesian version of backfitting at its core. As we demonstrate on several regression and classification benchmarks, Bayesian backfitting offers a compelling alternative to current regression methods, especially when the size and dimensionality of the data challenge computational resources.
Original languageEnglish
Title of host publicationProceedings of the twenty-first international conference on Machine learning
Number of pages1
ISBN (Print)1-58113-838-5
Publication statusPublished - 2004

Cite this