Deterministic annealing for semi-supervised kernel machines

An intuitive approach to utilizing unlabeled data in kernel-based classification algorithms is to simply treat unknown labels as additional optimization variables. For marginbased loss functions, one can view this approach as attempting to learn low-density separators. However, this is a hard optimization problem to solve in typical semisupervised settings where unlabeled data is abundant. The popular Transductive SVM algorithm is a label-switching-retraining procedure that is known to be susceptible to local minima. In this paper, we present a global optimization framework for semi-supervised Kernel machines where an easier problem is parametrically deformed to the original hard problem and minimizers are smoothly tracked. Our approach is motivated from deterministic annealing techniques and involves a sequence of convex optimization problems that are exactly and efficiently solved. We present empirical results on several synthetic and real world datasets that demonstrate the effecti...

• Type: Conference
• Year: 2006
• Where: ICML
• Authors: Vikas Sindhwani, S. Sathiya Keerthi, Olivier Chapelle
• Keywords: Additional Optimization Variables, Convex Optimization Problems, Hard Optimization Problem, ICML 2006, Machine Learning