Healing the relevance vector machine through augmentation

The Relevance Vector Machine (RVM) is a sparse approximate Bayesian kernel method. It provides full predictive distributions for test cases. However, the predictive uncertainties have the unintuitive property, that they get smaller the further you move away from the training cases. We give a thorough analysis. Inspired by the analogy to nondegenerate Gaussian Processes, we suggest augmentation to solve the problem. The purpose of the resulting model, RVM*, is primarily to corroborate the theoretical and experimental analysis. Although RVM* could be used in practical applications, it is no longer a truly sparse model. Experiments show that sparsity comes at the expense of worse predictive distributions. Bayesian inference based on Gaussian Processes (GPs) has become widespread in the machine learning community. However, their na?ive applicability is marred by computational constraints. A number of recent publications have addressed this issue by means of sparse approximations, although...

• Type: Conference
• Year: 2005
• Where: ICML
• Authors: Carl Edward Rasmussen, Joaquin Quiñonero Candela
• Keywords: Approximate Bayesian Kernel, ICML 2005, Machine Learning, Sparseness Arises, Sparseness Corresponds