Throttling I/O Streams to Accelerate File-IO Performance

To increase the scale and performance of scientific applications, scientists commonly distribute computation over multiple processors. Often without realizing it, file I/O is parallelized with the computation. An implication of this I/O parallelization is that multiple compute tasks are likely to concurrently access the I/O nodes of an HPC system. When a large number of I/O streams concurrently access an I/O node, I/O performance tends to degrade. In turn, this impacts application execution time. This paper presents experimental results that show that controlling the number of synchronous file-I/O streams that concurrently access an I/O node can enhance performance. We call this mechanism file-I/O stream throttling. The paper (1) describes this mechanism and demonstrates how it can be applied either at the application or system software layers, and (2) presents results of experiments driven by the cosmology application benchmark MADbench, executed on a variety of computing systems...

• Type: Conference
• Year: 2007
• Where: HPCC
• Authors: Seetharami R. Seelam, Andre Kerstens, Patricia J. Teller
• Keywords: Distributed And Parallel Computing, HPCC 2007, I/o Node, I/O Performance, file-i/o Streams