Efficient Data-Movement for Lightweight I/O

Efficient data movement is an important part of any highperformance I/O system, but it is especially critical for the current and next-generation of massively parallel processing (MPP) systems. In this paper, we discuss how the scale, architecture, and organization of current and proposed MPP systems impact the design of the data-movement scheme for the I/O system. We also describe and analyze the approach used by the Lightweight File Systems (LWFS) project, and we compare that approach to more conventional data-movement protocols used by small and mid-range clusters. Our results indicate that the datamovement strategy used by LWFS clearly outperforms conventional data-movement protocols, particularly as data sizes increase.

• Type: Conference
• Year: 2006
• Where: CLUSTER
• Authors: Ron Oldfield, Patrick Widener, Arthur B. Maccabe, Lee Ward, Todd Kordenbrock
• Keywords: CLUSTER 2006, Conventional Data-movement Protocols, Distributed And Parallel Computing, Efficient Data Movement, I/o System