AnthillSched: A Scheduling Strategy for Irregular and Iterative I/O-Intensive Parallel Jobs

Irregular and iterative I/O-intensive jobs need a different approach from parallel job schedulers. The focus in this case is not only the processing requirements anymore: memory, network and storage capacity must all be considered in making a scheduling decision. Job executions are irregular and data dependent, alternating between CPU-bound and I/O-bound phases. In this paper, we propose and implement a parallel job scheduling strategy for such jobs, called AnthillSched, based on a simple heuristic: we map the behavior of an parallel application with minimal resources as we vary its input parameters. From that mapping we infer the best scheduling for a certain set of input parameters given the available resources. To test and verify AnthillSched we used logs obtained from a real system executing data mining jobs. Our main contributions are the implementation of a parallel job scheduling strategy, called AnthillSched in a real system, and a performance analysis of AnthillSched, which a...

• Type: Conference
• Year: 2005
• Where: JSSPP
• Authors: Luís Fabrício Wanderley Góes, Pedro H. C. Guerra, Bruno Coutinho, Leonardo C. da Rocha, Wagner Meira Jr., Renato Ferreira, Dorgival Olavo Guedes Neto, Walfredo Cirne
• Keywords: Job Scheduling Strategy, JSSPP 2005, Parallel Job, Parallel Job Scheduling