Eliminating the middleman: peer-to-peer dataflow

Efficiently executing large-scale, data-intensive workflows such as Montage must take into account the volume and pattern of communication. When orchestrating data-centric workflows, centralised servers common to standard workflow systems can become a bottleneck to performance. However, standards-based workflow systems that rely on centralisation, e.g., Web service based frameworks, have many other benefits such as a wide user base and sustained support. This paper presents and evaluates a light-weight hybrid architecture which maintains the robustness and simplicity of centralised orchestration, but facilitates choreography by allowing services to exchange data directly with one another. Furthermore our architecture is standards compliment, flexible and is a non-disruptive solution; service definitions do not have to be altered prior to enactment. Our architecture could be realised within any existing workflow framework, in this paper, we focus on a Web service based framework. Takin...

• Type: Conference
• Year: 2008
• Where: HPDC
• Authors: Adam Barker, Jon B. Weissman, Jano I. van Hemert
• Keywords: Distributed And Parallel Computing, Fold Performance Benefit, HPDC 2008, Workflow, Workflow Systems