RFS: efficient and flexible remote file access for MPI-IO

Scientific applications often need to access remote file systems. Because of slow networks and large data size, however, remote I/O can become an even more serious performance bottleneck than local I/O performance. In this work, we present RFS, a high-performance remote I/O facility for ROMIO, which is a well-known MPI-IO implementation. Our simple, portable, and flexible design eliminates the shortcomings of previous remote I/O efforts. In particular, RFS improves the remote I/O performance by adopting active buffering with threads (ABT), which hides I/O cost by aggressively buffering the output data using available memory and performing background I/O using threads while computation is taking place. Our experimental results show that RFS with ABT can significantly reduce the remote I/O visible cost, achieving up to 92% of the theoretical peak throughput. The computation slowdown caused by concurrent I/O activities was 0.2

• Type: Conference
• Year: 2004
• Where: CLUSTER
• Authors: Jonghyun Lee, Robert B. Ross, Rajeev Thakur, Xiaosong Ma, Marianne Winslett
• Keywords: CLUSTER 2004, Distributed And Parallel Computing, I/O Performance, Remote I/o, Remote I/o Performance