Leveraging Non-Uniform Resources for Parallel Query Processing

Clusters are now composed of non-uniform nodes with different CPUs, disks or network cards so that customers can adapt the cluster configuration to the changing technologies and to their changing needs. Such modular clusters challenge parallel databases. The load balancing techniques used by existing parallel databases partition data across a set of nodes that all run the same relational operations. We show in this paper that this form of load balancing is ill suited for modular clusters because running the same operation on different subsets of the data does not fully utilize non-uniform hardware resources. We propose and evaluate new load balancing techniques that blend pipeline parallelism with intra-query parallelism. We consider relational operators as pipelines of fine-grained operations that can be located on different cluster nodes and executed in parallel on different data subsets to best exploit non-uniform resources. Our new techniques thus partition both operations and dat...

• Type: Conference
• Year: 2003
• Where: CCGRID
• Authors: Tobias Mayr, Philippe Bonnet, Johannes Gehrke, Praveen Seshadri
• Keywords: CCGRID 2003, Cluster Computing, Load Balancing, Load Balancing Techniques, Parallel Database