/scratch as a cache: rethinking HPC center scratch storage

To sustain emerging data-intensive scientific applications, High Performance Computing (HPC) centers invest a notable fraction of their operating budget on a specialized fast storage system, scratch space, which is designed for storing the data of currently running and soon-to-run HPC jobs. Instead, it is often used as a standard file system, wherein users arbitrarily store their data, without any consideration to the center’s overall performance. To remedy this, centers periodically scan the scratch in an attempt to purge transient and stale data. This practice of supporting a cache workload using a file system and disjoint tools for staging and purging results in suboptimal use of the scratch space. In this paper, we address the above issues by proposing a new perspective, where the HPC scratch space is treated as a cache, and data population, retention, and eviction tools are integrated with scratch management. In our approach, data is moved to the scratch space only when it n...

• Type: Conference
• Year: 2009
• Where: ICS
• Authors: Henry M. Monti, Ali Raza Butt, Sudharshan S. Vazhkudai
• Keywords: HPC Scratch, HPC Scratch Space, ICS 2009, Scratch Space, Theoretical Computer Science