Improving MPI Independent Write Performance Using A Two-Stage Write-Behind Buffering Method

Many large-scale production applications often have very long executions times and require periodic data checkpoints in order to save the state of the computation for program restart and/or tracing application progress. These write-only operations often dominate the overall application runtime, which makes them a good optimization target. Existing approaches for write-behind data buffering at the MPI I/O level have been proposed, but challenges still exist for addressing system-level I/O issues. We propose a twostage write-behind buffering scheme for handing checkpoint operations. The first-stage of buffering accumulates write data for better network utilization and the second-stage of buffering enables the alignment for the write requests to the file stripe boundaries. Aligned I/O requests avoid file lock contention that can seriously degrade I/O performance. We present our performance evaluation using BTIO benchmarks on both GPFS and Lustre file systems. With the twostage buffer...

• Type: Conference
• Year: 2007
• Where: IPPS
• Authors: Wei-keng Liao, Avery Ching, Kenin Coloma, Alok N. Choudhary, Mahmut T. Kandemir
• Keywords: Distributed And Parallel Computing, IPPS 2007, MPI I/O Level, Write-behind Buffering Scheme, Write-behind Data Buffering