Exploring many task computing in scientific workflows

One of the main advantages of using a scientific workflow management system (SWfMS) to orchestrate data flows among scientific activities is to control and register the whole workflow execution. The execution of activities within a workflow with high performance computing (HPC) presents challenges in SWfMS execution control. Current solutions leave the scheduling to the HPC queue system. Since the workflow execution engine does not run on remote clusters, SWfMS are not aware of the parallel strategy of the workflow execution. Consequently, remote execution control and provenance registry of the parallel activities is very limited from the SWfMS side. This work presents a set of components to be included on the workflow specification of any SWMfS to control parallelization of activities as MTC. In addition, these components can gather provenance data during remote workflow execution. Through these MTC components, the parallelization strategy can be registered and reused, and provenance...

• Type: Conference
• Year: 2009
• Where: SC
• Authors: Eduardo S. Ogasawara, Daniel de Oliveira, Fernando Seabra Chirigati, Carlos Eduardo Barbosa, Renato N. Elias, Vanessa P. Braganholo, Alvaro L. G. A. Coutinho, Marta Mattoso
• Keywords: Applied Computing, Execution Control, SC 2009, Scientific Workflows, Workflow Execution