Applying Twister to Scientific Applications

Many scientific applications suffer from the lack of a unified approach to support the management and efficient processing of large-scale data. The Twister MapReduce Framework, which not only supports the traditional MapReduce programming model but also extends it by allowing iterations, addresses these problems. This paper describes how Twister is applied to several kinds of scientific applications such as BLAST, MDS Interpolation and GTM Interpolation in a non-iterative style and to MDS without interpolation in an iterative style. The results show the applicability of Twister to data parallel and EM algorithms with small overhead and increased efficiency. Keywords-Twister; Iterative MapReduce; Cloud; Scientific Applications

• Type: Journal
• Year: 2010
• Where: CLOUDCOM
• Authors: Bingjing Zhang, Yang Ruan, Tak-Lon Wu, Judy Qiu, Adam Hughes, Geoffrey Fox
• Keywords: CLOUDCOM 2010, Distributed And Parallel Computing, Many Scientific Applications, Scientific Applications, Twister MapReduce Framework