CellMR: A framework for supporting mapreduce on asymmetric cell-based clusters

The use of asymmetric multi-core processors with onchip computational accelerators is becoming common in a variety of environments ranging from scientific computing to enterprise applications. The focus of current research has been on making efficient use of individual systems, and porting applications to asymmetric processors. In this paper, we take the next step by investigating the use of multi-corebased systems, especially the popular Cell processor, in a cluster setting. We present CellMR, an efficient and scalable implementation of the MapReduce framework for asymmetric Cell-based clusters. The novelty of CellMR lies in its adoption of a streaming approach to supporting MapReduce, and its adaptive resource scheduling schemes: Instead of allocating workloads to the components once, CellMR slices the input into small work units and streams them to the asymmetric nodes for efficient processing. Moreover, CellMR removes I/O bottlenecks by design, using a number of techniques, su...

• Type: Conference
• Year: 2009
• Where: IPPS
• Authors: M. Mustafa Rafique, Benjamin Rose, Ali Raza Butt, Dimitrios S. Nikolopoulos
• Keywords: Asymmetric Cell-based Clusters, Asymmetric Multi-core Processors, Asymmetric Processors, Distributed And Parallel Computing, IPPS 2009