Online and batch learning of pseudo-metrics

We describe and analyze an online algorithm for supervised learning of pseudo-metrics. The algorithm receives pairs of instances and predicts their similarity according to a pseudo-metric. The pseudo-metrics we use are quadratic forms parameterized by positive semi-definite matrices. The core of the algorithm is an update rule that is based on successive projections onto the positive semi-definite cone and onto half-space constraints imposed by the examples. We describe an efficient procedure for performing these projections, derive a worst case mistake bound on the similarity predictions, and discuss a dual version of the algorithm in which it is simple to incorporate kernel operators. The online algorithm also serves as a building block for deriving a large-margin batch algorithm. We demonstrate the merits of the proposed approach by conducting experiments on MNIST dataset and on document filtering.

• Type: Conference
• Year: 2004
• Where: ICML
• Authors: Shai Shalev-Shwartz, Yoram Singer, Andrew Y. Ng
• Keywords: Algorithm Receives Pairs, ICML 2004, Large-margin Batch Algorithm, Machine Learning, Online Algorithm