Damped Newton Algorithms for Matrix Factorization with Missing Data

The problem of low-rank matrix factorization in the presence of missing data has seen significant attention in recent computer vision research. The approach that dominates the literature is EM-like alternation of closed-form solutions for the two factors of the matrix. An obvious alternative is nonlinear optimization of both factors simultaneously, a strategy which has seen little published research. This paper provides a comprehensive comparison of the two strategies by evaluating previously published factorization algorithms as well as some second order methods not previously presented for this problem. We conclude that, although alternation approaches can be very quick, their propensity to glacial convergence in narrow valleys of the cost function means that averagecase performance is worse than second-order strategies. Further, we demonstrate the importance of two main observations: one, that schemes based on closed-form solutions alone are not suitable and that non-linear optimiz...

• Type: Conference
• Year: 2005
• Where: CVPR
• Authors: A. M. Buchanan, Andrew W. Fitzgibbon
• Keywords: Alternation Approaches, Computer Vision, CVPR 2005, Factorization Algorithms, Low-rank Matrix Factorization, Non-linear Optimization Strategies, Second-order Strategies