Sampling Methods for Unsupervised Learning

We present an algorithm to overcome the local maxima problem in estimating the parameters of mixture models. It combines existing approaches from both EM and a robust fitting algorithm, RANSAC, to give a data-driven stochastic learning scheme. Minimal subsets of data points, sufficient to constrain the parameters of the model, are drawn from proposal densities to discover new regions of high likelihood. The proposal densities are learnt using EM and bias the sampling toward promising solutions. The algorithm is computationally efficient, as well as effective at escaping from local maxima. We compare it with alternative methods, including EM and RANSAC, on both challenging synthetic data and the computer vision problem of alpha-matting.

• Type: Conference
• Year: 2004
• Where: NIPS
• Authors: Robert Fergus, Andrew Zisserman, Pietro Perona
• Keywords: Local Maxima, Local Maxima Problem, NIPS 2004, NIPS 2007, Proposal Densities