Distributed Stream Processing with DUP

This paper introduces the DUP System, a simple framework for parallel stream processing. The DUP System enables developers to compose applications from stages written in almost any programming language and to run distributed streaming applications across all POSIXcompatible platforms. Parallel applications written with the DUP System do not suffer from many of the problems that exist in traditional parallel languages. The DUP System includes a range of simple stages that serve as general-purpose building blocks for larger applications. This work describes the DUP Assembly language, the DUP architecture and some of the stages included in the DUP run-time library. We then present our experiences with parallelizing and distributing the ARB project, a package of tools for RNA/DNA sequence database handling and analysis.

• Type: Journal
• Year: 2010
• Where: NPC
• Authors: Kai Christian Bader, Tilo Eißler, Nathan S. Evans, Chris GauthierDickey, Christian Grothoff, Krista Grothoff, Jeff Keene, Harald Meier, Craig Ritzdorf, Matthew J. Rutherford
• Keywords: Distributed And Parallel Computing, DUP Assembly Language, DUP Run-time Library, DUP System, NPC 2010