Using Deep Belief Nets to Learn Covariance Kernels for Gaussian Processes

We show how to use unlabeled data and a deep belief net (DBN) to learn a good covariance kernel for a Gaussian process. We first learn a deep generative model of the unlabeled data using the fast, greedy algorithm introduced by [7]. If the data is high-dimensional and highly-structured, a Gaussian kernel applied to the top layer of features in the DBN works much better than a similar kernel applied to the raw input. Performance at both regression and classification can then be further improved by using backpropagation through the DBN to discriminatively fine-tune the covariance kernel.

• Type: Conference
• Year: 2007
• Where: NIPS
• Authors: Ruslan Salakhutdinov, Geoffrey E. Hinton
• Keywords: Covariance Kernel, Deep Belief Net, Information Technology, NIPS 2007, Unlabeled Data