Enabling personal clusters on demand for batch resources using commodity software

Providing QoS (Quality of Service) in batch resources against the uncertainty of resource availability due to the space-sharing nature of scheduling policies is a critical capability required for high-performance computing. This paper introduces a technique called personal cluster which reserves a partition of batch resources on user’s demand in a best-effort manner. A personal cluster provides a private cluster dedicated to the user during a user-specified time period by installing a user-level resource manager on the resource partition. This technique not only enables cost-effective resource utilization and efficient task management but also provides the user a uniform interface to heterogeneous resources regardless of local resource management software. A prototype implementation using a PBS batch resource manager and Globus Toolkits based on Web Services shows that the overhead of instantiating a personal cluster of medium size is small, which is just about 1 minute for a person...

• Type: Conference
• Year: 2008
• Where: IPPS
• Authors: Yang-Suk Kee, Carl Kesselman, Daniel Nurmi, Richard Wolski
• Keywords: Batch Resource, Distributed And Parallel Computing, IPPS 2008, Personal Cluster, Resource