Implicit Regularization in Variational Bayesian Matrix Factorization

Matrix factorization into the product of lowrank matrices induces non-identifiability, i.e., the mapping between the target matrix and factorized matrices is not one-to-one. In this paper, we theoretically investigate the influence of non-identifiability on Bayesian matrix factorization. More specifically, we show that a variational Bayesian method involves regularization effect even when the prior is non-informative, which is intrinsically different from the maximum a posteriori approach. We also extend our analysis to empirical Bayes scenarios where hyperparameters are also learned from data.

• Type: Journal
• Year: 2010
• Where: ICML
• Authors: Shinichi Nakajima, Masashi Sugiyama
• Keywords: Bayesian Matrix Factorization, ICML 2010, Machine Learning, Matrix Factorization, Variational Bayesian Method