Efficient Parallel Hierarchical Clustering

Hierarchical agglomerative clustering (HAC) is a common clustering method that outputs a dendrogram showing all N levels of agglomerations where N is the number of objects in the data set. High time and memory complexities are some of the major bottlenecks in its application to real-world problems. In the literature parallel algorithms are proposed to overcome these limitations. But, as this paper shows, existing parallel HAC algorithms are inefficient due to ineffective partitioning of the data. We first show how HAC follows a rule where most agglomerations have very small dissimilarity and only a small portion towards the end have large dissimilarity. Partially overlapping partitioning (POP) exploits this principle and obtains efficient yet accurate HAC algorithms. The total number of dissimilarities is reduced by a factor close to the number of cells in the partition. We present pPOP, the parallel version of POP, that is implemented on a shared memory multiprocessor architecture. Ex...

• Type: Conference
• Year: 2004
• Where: EUROPAR
• Authors: Manoranjan Dash, Simona Petrutiu, Peter Scheuermann
• Keywords: Distributed And Parallel Computing, EUROPAR 2004, HAC Algorithms, Parallel Algorithms, Parallel Hac Algorithms