Accurately measuring collective operations at massive scale

Accurate, reproducible and comparable measurement of collective operations is a complicated task. Although Different measurement schemes are implemented in wellknown benchmarks, many of these schemes introduce different systematic errors in their measurements. We characterize these errors and select a window-based approach as the most accurate method. However, this approach complicates measurements significantly and introduces a clock synchronization as a new source of systematic errors. We analyze approaches to avoid or correct those errors and develop a scalable synchronization scheme to conduct benchmarks on massively parallel systems. Our results are compared to the window-based scheme implemented in the SKaMPI benchmarks and show a reduction of the synchronization overhead by a factor of 16 on 128 processes.

• Type: Conference
• Year: 2008
• Where: IPPS
• Authors: Torsten Hoefler, Timo Schneider, Andrew Lumsdaine
• Keywords: Distributed And Parallel Computing, IPPS 2008, Scalable Synchronization Scheme, Systematic Errors, Wellknown Benchmarks