An alternative view of variational Bayes and asymptotic approximations of free energy

Bayesian learning, widely used in many applied data-modeling problems, is often accomplished with approximation schemes because it requires intractable computation of the posterior distributions. In this study, we focus on two approximation methods, variational Bayes and local variational approximation. We show that the variational Bayes approach for statistical models with latent variables can be viewed as a special case of local variational approximation, where the log-sum-exp function is used to form the lower bound of the log-likelihood. The minimum variational free energy, the objective function of variational Bayes, is analyzed and related to the asymptotic theory of Bayesian learning. This analysis additionally implies a relationship between the generalization performance of the variational Bayes approach and the minimum variational free energy.

• Type: Journal
• Year: 2012
• Where: ML
• Authors: Kazuho Watanabe
• Keywords: Approximation Methods, Approximation Schemes, Generalization Performance, Machine Learning, ML 2012