Performance characterization and optimization of parallel I/O on the Cray XT

This paper presents an extensive characterization, tuning, and optimization of parallel I/O on the Cray XT supercomputer, named Jaguar, at Oak Ridge National Laboratory. We have characterized the performance and scalability for different levels of storage hierarchy including a single Lustre object storage target, a single S2A storage couplet, and the entire system. Our analysis covers both data- and metadata-intensive I/O patterns. In particular, for small, non-contiguous dataintensive I/O on Jaguar, we have evaluated several parallel I/O techniques, such as data sieving and twophase collective I/O, and shed light on their effectiveness. Based on our characterization, we have demonstrated that it is possible, and often prudent, to improve the I/O performance of scientific benchmarks and applications by tuning and optimizing I/O. For example, we demonstrate that the I/O performance of the S3D combustion application can be improved at large scale by tuning the I/O system to avoid a band...

• Type: Conference
• Year: 2008
• Where: IPPS
• Authors: Weikuan Yu, Jeffrey S. Vetter, Sarp Oral
• Keywords: Collective I/o, Distributed And Parallel Computing, I/O Performance, IPPS 2008, Parallel I/o