Experiences with Fine-Grained Distributed Supercomputing on a 10G Testbed

This paper shows how lightpath-based networks can allow challenging, fine-grained parallel supercomputing applications to be run on a grid, using parallel retrograde analysis on DAS-3 as a case study. Detailed performance analysis shows that several problems arise that are not present on tightly-coupled systems like clusters. In particular, flow control, asynchronous communication, and hostlevel communication overheads become new obstacles. By optimizing these aspects, however, a 10G grid can obtain high performance for this type of communication-intensive application. The class of large-scale distributed applications suitable for running on a grid is therefore larger than previously thought realistic.

• Type: Conference
• Year: 2008
• Where: CCGRID
• Authors: Kees Verstoep, Jason Maassen, Henri E. Bal, John W. Romein
• Keywords: CCGRID 2008, Cluster Computing, Detailed Performance Analysis, Parallel Retrograde Analysis, Parallel Supercomputing Applications