An Adaptive, Load Balancing Parallel Join Algorithm

Many parallel join algorithms have been proposed in the last several years. However, most of these algorithms require that the amount of data to be joined is known in advance in order to choose the proper number of join processors. This is an unrealistic assumption because data sizes are typically unknown, and are notoriously hard to estimate. We present an adaptive, load-balancing parallel join algorithm called PJLH to address this problem. PJLH eciently adapts itself to use additional processors if the amount of data is larger than expected. Furthermore, while adapting, it ensures a good load balancing of data across the processors. We have implemented and analyzed PJLH on a main memory database system implemented on a cluster of workstations. We show that PJLH is nearly as ecient as an optimal algorithmwhen the amount of data is known in advance. Furthermore, we show that PJLH eciently adapts to use additional join processors when necessary, while maintaining a balanced load. This ...

• Type: Conference
• Year: 1994
• Where: COMAD
• Authors: Minesh B. Amin, Donovan A. Schneider, V. Singh
• Keywords: COMAD 1994, COMAD 2008, Join Processors, Many Parallel Join, Parallel Join Algorithm