Sciweavers

EUROSYS
2007
ACM

Dryad: distributed data-parallel programs from sequential building blocks

14 years 9 months ago
Dryad: distributed data-parallel programs from sequential building blocks
Dryad is a general-purpose distributed execution engine for coarse-grain data-parallel applications. A Dryad application combines computational “vertices” with communication “channels” to form a dataflow graph. Dryad runs the application by executing the vertices of this graph on a set of available computers, communicating as appropriate through files, TCP pipes, and shared-memory FIFOs. The vertices provided by the application developer are quite simple and are usually written as sequential programs with no thread creation or locking. Concurrency arises from Dryad scheduling vertices to run simultaneously on multiple computers, or on multiple CPU cores within a computer. The application can discover the size and placement of data at run time, and modify the graph as the computation progresses to make efficient use of the available resources. Dryad is designed to scale from powerful multi-core single computers, through small clusters of computers, to data centers with thousa...
Michael Isard, Mihai Budiu, Yuan Yu, Andrew Birrel
Added 10 Mar 2010
Updated 10 Mar 2010
Type Conference
Year 2007
Where EUROSYS
Authors Michael Isard, Mihai Budiu, Yuan Yu, Andrew Birrell, Dennis Fetterly
Comments (0)