Using early phase termination to eliminate load imbalances at barrier synchronization points

We present a new technique, early phase termination, for eliminating idle processors in parallel computations that use barrier synchronization. This technique simply terminates each parallel phase as soon as there are too few remaining tasks to keep all of the processors busy. Although this technique completely eliminates the idling that would otherwise occur at barrier synchronization points, it may also change the computation and therefore the result that the computation produces. We address this issue by providing probabilistic distortion models that characterize how the use of early phase termination distorts the result that the computation produces. Our experimental results show that for our set of benchmark applications, 1) early phase termination can improve the performance of the parallel computation, 2) the distortion is small (or can be made to be small with the use of an appropriate compensation technique) and 3) the distortion models provide accurate and tight distortion b...

• Type: Conference
• Year: 2007
• Where: OOPSLA
• Authors: Martin C. Rinard
• Keywords: Distortion, Early Phase Termination, OOPSLA 2007, Parallel Computations