On the equivalence between Non-negative Matrix Factorization and Probabilistic Latent Semantic Indexing

Non-negative Matrix Factorization (NMF) and Probabilistic Latent Semantic Indexing (PLSI) have been successfully applied to document clustering recently. In this paper, we show that PLSI and NMF (with the I-divergence objective function) optimize the same objective function, although PLSI and NMF are different algorithms as verified by experiments. This provides a theoretical basis for a new hybrid method that runs PLSI and NMF alternatively, each jumping out of the local minima of the other method successively, thus achieving a better final solution. Extensive experiments on five real-life datasets show relations between NMF and PLSI, and indicate that the hybrid method leads to significant improvements over NMF-only or PLSI-only methods. We also show that at first-order approximation, NMF is identical to the 2-statistic. c 2008 Published by Elsevier B.V.

• Type: Journal
• Year: 2008
• Where: CSDA
• Authors: Chris H. Q. Ding, Tao Li, Wei Peng
• Keywords: CSDA 2008, Latent Semantic Indexing, Non-Negative Matrix Factorization, PLSI