Optimal Manifold Representation of Data: An Information Theoretic Approach

We introduce an information theoretic method for nonparametric, nonlinear dimensionality reduction, based on the infinite cluster limit of rate distortion theory. By constraining the information available to manifold coordinates, a natural probabilistic map emerges that assigns original data to corresponding points on a lower dimensional manifold. With only the information-distortion trade off as a parameter, our method determines the shape of the manifold, its dimensionality, the probabilistic map and the prior that provide optimal description of the data. 1 A simple example Some data sets may not be as complicated as they appear. Consider the set of points on a

• Type: Conference
• Year: 2003
• Where: NIPS
• Authors: Denis V. Chigirev, William Bialek
• Keywords: Lower Dimensional Manifold, NIPS 2003, NIPS 2007, Nonlinear Dimensionality Reduction, Probabilistic Map