High performance file I/O for the Blue Gene/L supercomputer

Parallel I/O plays a crucial role for most data-intensive applications running on massively parallel systems like Blue Gene/L that provides the promise of delivering enormous computational capability. We designed and implemented a highly scalable parallel file I/O architecture for Blue Gene/L, which leverages the benefit of the hierarchical and functional partitioning design of the system software with separate computational and I/O cores. The architecture exploits the scalability aspect of GPFS (General Parallel File System) at the backend, while using MPI I/O as an interface between the application I/O and the file system. We demonstrate the impact of our high performance I/O solution for Blue Gene/L with a comprehensive evaluation that consists of a number of widely used parallel I/O benchmarks and I/O intensive applications. Our design and implementation is not only able to deliver at least one order of magnitude speed up in terms of I/O bandwidth for a real-scale appli

• Type: Conference
• Year: 2006
• Where: HPCA
• Authors: Hao Yu, Ramendra K. Sahoo, C. Howson, G. Almasi, José G. Castaños, Manish Gupta, José E. Moreira, J. J. Parker, Thomas Engelsiepen, Robert B. Ross, Rajeev Thakur, Robert Latham, William D. Gropp
• Keywords: Computer Architecture, HPCA 2006, I/O Intensive Applications, Parallel File I/o, Parallel I/o Benchmarks