Scheduling Mixed Workloads in Multi-grids: The Grid Execution Hierarchy

Consider a workload in which massively parallel tasks that require large resource pools are interleaved with short tasks that require fast response but consume fewer resources. We aim at achieving high throughput and short response time when scheduling such a workload over a set of uncoordinated grids of varying sizes and performance characteristics. We propose the concept of a grid execution hierarchy, where available grids are sorted according to their size, and the execution overheads increase with the size of the grids. We devise a scheduling algorithm for this execution hierarchy of grids by adapting the multilevel feedback queue approach to a multi-grid environment. The algorithm finds a grid of the size, availability, and overhead that best matches a task’s resource requirements and expected turnaround time. Our approach is inspired by the Shortest Processing Time First policy (SPTF), in the sense that the task’s processing demands are constantly reevaluated during its run...

• Type: Conference
• Year: 2006
• Where: HPDC
• Authors: Mark Silberstein, Dan Geiger, Assaf Schuster, Miron Livny
• Keywords: Distributed And Parallel Computing, Execution Hierarchy, Grids, HPDC 2006, Parallel Tasks