A Heuristic Algorithm for Mapping Communicating Tasks on Heterogeneous Resources

A heuristic algorithm that maps data-processing tasks onto heterogeneous resources (i.e., processors and links of various capacities) is presented. The algorithm tries to achieve a good throughput of the whole data-processing pipeline, taking both parallelism (load balance) and communication volume (locality) into account. It performs well both under compute-intensive and communication-intensive conditions. When all tasks/processors are of the same size and communication is negligible, it quickly distributes the compute load over processors and finds the optimal mapping. As communication becomes significant and reveals as a bottleneck, it trades parallelism for reduction of communication traffic. Experimental results using a topology generator that models the Internet show that it performs significantly better than communication-ignorant schedulers.

• Type: Conference
• Year: 2000
• Where: HCW
• Authors: Kenjiro Taura, Andrew A. Chien
• Keywords: Communication, Data-processing Pipeline, Data-processing Tasks, Distributed And Parallel Computing, HCW 2000