Sciweavers

JSSPP
2009
Springer

Job Admission and Resource Allocation in Distributed Streaming Systems

14 years 7 months ago
Job Admission and Resource Allocation in Distributed Streaming Systems
Abstract. This paper describes a new and novel scheme for job admission and resource allocation employed by the SODA scheduler in System S. Capable of processing enormous quantities of streaming data, System S is a large-scale, distributed stream processing system designed to handle complex applications. The problem of scheduling in distributed, stream-based systems is quite unlike that in more traditional systems. And the requirements for System S, in particular, are more stringent than one might expect even in a “standard” stream-based design. For example, in System S, the offered load is expected to vastly exceed system capacity. So a careful job admission scheme is essential. The jobs in System S are essentially directed graphs, with software “processing elements” (PEs) as vertices and data streams as edges connecting the PEs. The jobs themselves are often heavily interconnected. Thus resource allocation of individual PEs must be done carefully in order to balance the flo...
Joel L. Wolf, Nikhil Bansal, Kirsten Hildrum, Suja
Added 27 May 2010
Updated 27 May 2010
Type Conference
Year 2009
Where JSSPP
Authors Joel L. Wolf, Nikhil Bansal, Kirsten Hildrum, Sujay Parekh, Deepak Rajan, Rohit Wagle, Kun-Lung Wu
Comments (0)