SMEM Algorithm for Mixture Models

When learning a mixture model, we suffer from the local optima and model structure determination problems. In this paper, we present a method for simultaneously solving these problems based on the variational Bayesian (VB) framework. First, in the VB framework, we derive an objective function that can simultaneously optimize both model parameter distributions and model structure. Next, focusing on mixture models, we present a deterministic algorithm to approximately optimize the objective function by using the idea of the split and merge operations which we previously proposed within the maximum likelihood framework. Then, we apply the method to mixture of expers (MoE) models to experimentally show that the proposed method can find the optimal number of experts of a MoE while avoiding local maxima. q 2002 Elsevier Science Ltd. All rights reserved.

• Type: Conference
• Year: 1998
• Where: NIPS
• Authors: Naonori Ueda, Ryohei Nakano, Zoubin Ghahramani, Geoffrey E. Hinton
• Keywords: Mixture Models, Model Structure, NIPS 1998, NIPS 2007, Objective Function