A Compilation Framework for Distributed Memory Parallelization of Data Mining Algorithms

With the availability of large datasets in a variety of scientific and commercial domains, data mining has emerged as an important area within the last decade. Data mining techniques focus on finding novel and useful patterns or models from large datasets. Because of the volume of the data to be analyzed, the amount of computation involved, and the need for rapid or even interactive analysis, data mining applications require the use of parallel machines. We believe that parallel compilation technology can be used for providing high-level language support for carrying out data mining implementations. Our study of a variety of popular data mining techniques has shown that they can be parallelized in a similar fashion. In our previous work, we have developed a middleware system that exploits this similarity to support distributed memory parallelization and execution on disk-resident datasets. This paper focuses on developing a data parallel language interface for using our middleware�...

• Type: Conference
• Year: 2003
• Where: IPPS
• Authors: Xiaogang Li, Ruoming Jin, Gagan Agrawal
• Keywords: Data Mining, Data Mining Techniques, Data Parallel, Distributed And Parallel Computing, IPPS 2003