Designing Accelerator-Based Distributed Systems for High Performance

Abstract--Multi-core processors with accelerators are becoming commodity components for high-performance computing at scale. While accelerator-based processors have been studied in some detail, the design and management of clusters based on these processors have not received the same focus. In this paper, we present an exploration of four design and resource management alternatives, which can be used on largescale asymmetric clusters with accelerators. Moreover, we adapt the popular MapReduce programming model to our proposed configurations. We enhance MapReduce with new dynamic data streaming and workload scheduling capabilities, which enable application writers to use asymmetric acceleratorbased clusters without being concerned with the capabilities of individual components. We present an evaluation of the presented designs in a physical setting and show that our designs can provide significant performance advantages. Compared to a standard static MapReduce design, we achieve 62.5%, ...

• Type: Conference
• Year: 2010
• Where: CCGRID
• Authors: M. Mustafa Rafique, Ali Raza Butt, Dimitrios S. Nikolopoulos
• Keywords: Asymmetric Clusters, CCGRID 2010, Distributed And Parallel Computing, General-purpose Resources, Largescale Asymmetric Clusters