PLDA: Parallel Latent Dirichlet Allocation for Large-Scale Applications

Abstract. This paper presents PLDA, our parallel implementation of Latent Dirichlet Allocation on MPI and MapReduce. PLDA smooths out storage and computation bottlenecks and provides fault recovery for lengthy distributed computations. We show that PLDA can be applied to large, real-world applications and achieves good scalability. We have released MPI-PLDA to open source at http://code.google.com/p/plda under the Apache License.

• Type: Conference
• Year: 2009
• Where: AAIM
• Authors: Yi Wang, Hongjie Bai, Matt Stanton, Wen-Yen Chen, Edward Y. Chang
• Keywords: AAIM 2009, Latent Dirichlet Allocation, Lengthy Distributed Computations, Paper Presents Plda