Deterministic Annealing for Semi-Supervised Structured Output Learning

In this paper we propose a new approach for semi-supervised structured output learning. Our approach uses relaxed labeling on unlabeled data to deal with the combinatorial nature of the label space and further uses domain constraints to guide the learning. Since the overall objective is non-convex, we alternate between the optimization of the model parameters and the label distribution of unlabeled data. The alternating optimization coupled with deterministic annealing helps us achieve better local optima and as a result our approach leads to better constraint satisfaction during inference. Experimental results on sequence labeling benchmarks show superior performance of our approach compared to Constraint Driven Learning (CoDL) and Posterior Regularization (PR).

• Type: Journal
• Year: 2012
• Where: JMLR
• Authors: Paramveer S. Dhillon, S. Sathiya Keerthi, Kedar Bellare, Olivier Chapelle, Sundararajan Sellamanickam
• Keywords: Constraint Satisfaction, Domain Constraints, JMLR 2012, Label Space, Programming Languages