Faster Rates for training Max-Margin Markov Networks

Structured output prediction is an important machine learning problem both in theory and practice, and the max-margin Markov network (M3 N) is an effective approach. All state-of-the-art algorithms for optimizing M3 N objectives take at least O(1/ ) number of iterations to find an accurate solution. Recent results in structured optimization suggest that faster rates are possible by exploiting the structure of the objective function. Towards this end Nesterov (2005b) proposed an excessive gap reduction technique based on Euclidean projections which converges in O(1/ ) iterations on strongly convex functions. Unfortunately when applied to M3 Ns, this approach does not admit graphical model factorization which, as in many existing algorithms, is crucial for keeping the cost per iteration tractable. In this paper, we present a new excessive gap reduction technique based on Bregman projections which admits graphical model factorization naturally, and converges in O(1/ ) iterations. Compa...

• Type: Journal
• Year: 2010
• Where: CORR
• Authors: Xinhua Zhang, Ankan Saha, S. V. N. Vishwanathan
• Keywords: CORR 2010, Education, Excessive Gap Reduction, Gap Reduction Technique, Graphical Model Factorization