A fast topology inference: a building block for network-aware parallel processing

Adapting to the network is the key to achieving high performance for communication-intensive applications, including scientific computing, data intensive computing, and multicast, especially in Grid environments. This paper investigates an approach of representing network as a tree of participating hosts and switches matching or approximating their physical topology, and describes a fast, non-intrusive, and portable algorithm for inferring such a topology. This representation and the proposed inference algorithm serves as a key to building network-aware applications in a portable manner. The algorithm is based solely on RTTs of small packets between end hosts; it does not rely on popular but not universally available protocols such as traceroute and SNMP. Another benefit is that it can handle all layers of network uniformly without any a priori knowledge of cluster configurations. The required number of measurements is O(Nd) in certain idealizing assumptions made for the purpose of...

• Type: Conference
• Year: 2007
• Where: HPDC
• Authors: Tatsuya Shirai, Hideo Saito, Kenjiro Taura
• Keywords: Building Network-aware Applications, Distributed And Parallel Computing, HPDC 2007, Inference Algorithm, Network-aware Applications