Global Coordination of Local Linear Models

High dimensional data that lies on or near a low dimensional manifold can be described by a collection of local linear models. Such a description, however, does not provide a global parameterization of the manifold--arguably an important goal of unsupervised learning. In this paper, we show how to learn a collection of local linear models that solves this more difficult problem. Our local linear models are represented by a mixture of factor analyzers, and the "global coordination" of these models is achieved by adding a regularizing term to the standard maximum likelihood objective function. The regularizer breaks a degeneracy in the mixture model's parameter space, favoring models whose internal coordinate systems are aligned in a consistent way. As a result, the internal coordinates change smoothly and continuously as one traverses a connected path on the manifold--even when the path crosses the domains of many different local models. The regularizer takes the form of...

• Type: Conference
• Year: 2001
• Where: NIPS
• Authors: Sam T. Roweis, Lawrence K. Saul, Geoffrey E. Hinton
• Keywords: Local Linear Models, Low Dimensional Manifold, NIPS 2001, NIPS 2007, Pixel Intensities