Toward Learning Gaussian Mixtures with Arbitrary Separation

In recent years analysis of complexity of learning Gaussian mixture models from sampled data has received significant attention in computational machine learning and theory communities. In this paper we present the first result showing that polynomial time learning of multidimensional Gaussian Mixture distributions is possible when the separation between the component means is arbitrarily small. Specifically, we present an algorithm for learning the parameters of a mixture of k identical spherical Gaussians in n-dimensional space with an arbitrarily small separation between the components, which is polynomial in dimension, inverse component separation and other input parameters for a fixed number of components k. The algorithm uses a projection to k dimensions and then a reduction to the 1-dimensional case. It relies on a theoretical analysis showing that two 1-dimensional mixtures whose densities are close in the L2 norm must have similar means and mixing coefficients. To produce the...

• Type: Journal
• Year: 2010
• Where: COLT
• Authors: Mikhail Belkin, Kaushik Sinha
• Keywords: Arbitrarily Small, COLT 2010, Gaussian Mixture, L2 Norm, Machine Learning