Winnowing subspaces

We generalize the Winnow algorithm for learning disjunctions to learning subspaces of low rank. Subspaces are represented by symmetric projection matrices. The online algorithm maintains its uncertainty about the hidden low rank projection matrix as a symmetric positive definite matrix. This matrix is updated using a version of the Matrix Exponentiated Gradient algorithm that is based on matrix exponentials and matrix logarithms. As in the case of the Winnow algorithm, the bounds are logarithmic in the dimension n of the problem, but linear in the rank r of the hidden subspace. We show that the algorithm can be adapted to handle arbitrary matrices of any dimension via a reduction.

• Type: Conference
• Year: 2007
• Where: ICML
• Authors: Manfred K. Warmuth
• Keywords: ICML 2007, Machine Learning, Positive Definite Matrix, Rank Projection Matrix, Winnow Algorithm