Non-negative Matrix Factorization (NMF, [5]) and Probabilistic Latent Semantic Analysis (PLSA, [4]) have been successfully applied to a number of text analysis tasks such as document clustering. Despite their different inspirations, both methods are instances of multinomial PCA [1]. We further explore this relationship and first show that PLSA solves the problem of NMF with KL divergence, and then explore the implications of this relationship. Categories and Subject Descriptors H.3.3 [Information Search and Retrieval]: Clustering; I.5.3 [Clustering]: Algorithms General Terms Algorithms Keywords Document clustering, probabilistic models, PLSA, NMF