Factorized Asymptotic Bayesian Inference for Mixture Modeling

This paper proposes a novel Bayesian approximation inference method for mixture modeling. Our key idea is to factorize marginal log-likelihood using a variational distribution over latent variables. An asymptotic approximation, a factorized information criterion (FIC), is obtained by applying the Laplace method to each of the factorized components. In order to evaluate FIC, we propose factorized asymptotic Bayesian inference (FAB), which maximizes an asymptotically-consistent lower bound of FIC. FIC and FAB have several desirable properties: 1) asymptotic consistency with the marginal log-likelihood, 2) automatic component selection on the basis of an intrinsic shrinkage mechanism, and 3) parameter identifiability in mixture modeling. Experimental results show that FAB outperforms state-of-the-art VB methods.

• Type: Journal
• Year: 2012
• Where: JMLR
• Authors: Ryohei Fujimaki, Satoshi Morinaga
• Keywords: Asymptotic Approximation, Asymptotic Consistency, Inference Method, JMLR 2012, Programming Languages