A MapReduce-Enabled Scientific Workflow Composition Framework

MapReduce has recently gained a lot of attention as a parallel programming model for scalable data-intensive business and scientific analysis. In order to benefit from this powerful programming model in a scientific workflow environment, we propose a MapReduce-enabled scientific workflow composition framework consisting of: i) a dataflowbased scientific workflow model that separates the declaration of the workflow interface from the definition of its functional body; ii) a set of dataflow constructs, including Map, Reduce, Loop, and Conditional, and their composition semantics to enable MapReduce-style scientific workflows; iii) an XML-based scientific workflow specification language, called WSL, in which both Map and Reduce are fully composable with other dataflow constructs in both flat and hierarchical manners. Besides leveraging the power of MapReduce to the workflow level, our workflow composition framework is unique in that workflows are the only operands for composition; in thi...

• Type: Journal
• Year: 2009
• Where: ICWS
• Authors: Xubo Fei, Shiyong Lu, Cui Lin
• Keywords: ICWS 2009, Internet Technology, Scientific Workflow, Workflow Composition Framework, Workflows