Dynamic Load-Balanced Multicast for Data-Intensive Applications on Clouds

Data-intensive parallel applications on clouds need to deploy large data sets from the cloud's storage facility to all compute nodes as fast as possible. Many multicast algorithms have been proposed for clusters and grid environments. The most common approach is to construct one or more spanning trees based on the network topology and network monitoring data in order to maximize available bandwidth and avoid bottleneck links. However, delivering optimal performance becomes difficult once the available bandwidth changes dynamically. In this paper, we focus on Amazon EC2/S3 (the most commonly used cloud platform today) and propose two high performance multicast algorithms. These algorithms make it possible to efficiently transfer large amounts of data stored in Amazon S3 to multiple Amazon EC2 nodes. The three salient features of our algorithms are (1) to construct an overlay network on clouds without network topology information, (2) to optimize the total throughput dynamically, an...

• Type: Journal
• Year: 2010
• Where: CCGRID
• Authors: Tatsuhiro Chiba, Mathijs den Burger, Thilo Kielmann, Satoshi Matsuoka
• Keywords: Algorithms, Available Bandwidth, CCGRID 2010, Distributed And Parallel Computing, Network Topology