High Performance Relay Mechanism for MPI Communication Libraries Run on Multiple Private IP Address Clusters

We have been developing a Grid-enabled MPI communication library called GridMPI, which is designed to run on multiple clusters connected to a wide-area network. Some of these clusters may use private IP addresses. Therefore, some mechanism to enable communication between private IP address clusters is required. Such a mechanism should be widely adoptable, and should provide high communication performance. In this paper, we propose a message relay mechanism to support private IP address clusters in the manner of the Interoperable MPI (IMPI) standard. Therefore, any MPI implementations which follow the IMPI standard can communicate with the relay. Furthermore, we also propose a trunking method in which multiple pairs of relay nodes simultaneously communicate between clusters to improve the available communication bandwidth. While the relay mechanism introduces an one-way latency of about 25 sec, the extra overhead is negligible, since the communication latency through a wide area network...

• Type: Conference
• Year: 2008
• Where: CCGRID
• Authors: Ryousei Takano, Motohiko Matsuda, Tomohiro Kudoh, Yuetsu Kodama, Fumihiro Okazaki, Yutaka Ishikawa, Yasufumi Yoshizawa
• Keywords: CCGRID 2008, Distributed And Parallel Computing, IP Address Clusters, Private Ip, Private Ip Address