External Memory Algorithms

Abstract. Data sets in large applications are often too massive to t completely inside the computer's internal memory. The resulting input output communication or I O between fast internal memory and slower external memory such as disks can be a major performance bottleneck. In this paper, we survey the state of the art in the design and analysis of external memory algorithms and data structures which are sometimes referred to as EM" or I O" or out-of-core" algorithms and data structures. EM algorithms and data structures are often designed and analyzed using the parallel disk model PDM. The three machine-independent measures of performance in PDM are the number of I O operations, the CPU time, and the amount of disk space. PDM allows for multiple disks or disk arrays and parallel CPUs, and it can be generalized to handle tertiary storage and hierarchical memory. We discuss several important paradigms for how to solve batched and online problems e ciently in externa...

• Type: Conference
• Year: 1998
• Where: ESA
• Authors: Jeffrey Scott Vitter
• Keywords: Algorithms, Data Structures, ESA 1998, External Memory, Internal Memory