An Implementation and Evaluation of Client-Side File Caching for MPI-IO

Client-side file caching has long been recognized as a file system enhancement to reduce the amount of data transfer between application processes and I/O servers. However, caching also introduces cache coherence problems when a file is simultaneously accessed by multiple processes. Existing coherence controls tend to treat the client processes independently and ignore the aggregate I/O access pattern. This causes a serious performance degradation for parallel I/O applications. In our earlier work, we proposed a caching system that enables cooperation among application processes in performing client-side file caching. The caching system has since been integrated into the MPI-IO library. In this paper we discuss our new implementation and present an extended performance evaluation on GPFS and Lustre parallel file systems. In addition to comparing our methods to traditional approaches, we examine the performance of MPI-IO caching under direct I/O mode to bypass the underlying file...

• Type: Conference
• Year: 2007
• Where: IPPS
• Authors: Wei-keng Liao, Avery Ching, Kenin Coloma, Alok N. Choudhary, Lee Ward
• Keywords: Application Processes, Caching, Client-side file Caching, Distributed And Parallel Computing, IPPS 2007