Out-of-Core Wavefront Computations with Reduced Synchronization

Matrix computation algorithms often exhibit dependencies between neighboring elements inside loop nests such that the frontier between computed elements and those to be computed wanders in form of a ‘wave’ through the matrix. Macro-pipelining techniques can achieve an efficient parallelization of such algorithms by overlapping communication and computation. Usually these techniques are limited to situations where all the data to be processed fits into main memory, whereas for larger data the I/O usage pattern for external storage requires special attention. The work [5] presented a first extension of the wavefront framework to these so-called out-of-core problems. The present paper proposes a redesign of their algorithm that minimizes both overhead and perturbations coming from communications. To tackle the issue of non-contiguous I/O, we also propose an optimized data layout. These two major modifications of the original algorithm eventually allow us to present a third improv...

• Type: Conference
• Year: 2008
• Where: PDP
• Authors: Pierre-Nicolas Clauss, Jens Gustedt, Frédéric Suter
• Keywords: Distributed And Parallel Computing, I/O Usage Pattern, Matrix Computation Algorithms, PDP 2008, So-called Out-of-core Problems