Variational Bayes Estimation of Mixing Coefficients

We investigate theoretically some properties of variational Bayes approximations based on estimating the mixing coefficients of known densities. We show that, with probability 1 as the sample size n grows large, the iterative algorithm for the variational Bayes approximation converges locally to the maximum likelihood estimator at the rate of O(1/n). Moreover, the variational posterior distribution for the parameters is shown to be asymptotically normal with the same mean but a different covariance matrix compared with those for the maximum likelihood estimator. Furthermore we prove that the covariance matrix from the variational Bayes approximation is `too small' compared with that for the MLE, so that resulting interval estimates for the parameters will be unrealistically narrow. Key words: Mixture model, Maximum likelihood, Variational Bayes, Local convergence, Asymptotic normality, Fisher information

• Type: Conference
• Year: 2004
• Where: DSMML
• Authors: Bo Wang 0002, D. M. Titterington
• Keywords: DSMML 2004, Machine Learning, Maximum Likelihood, Variational Bayes, Variational Bayes Approximation