Using a market economy to provision compute resources across planet-wide clusters

—We present a practical, market-based solution to the resource provisioning problem in a set of heterogeneous resource clusters. We focus on provisioning rather than immediate scheduling decisions to allow users to change long-term job specifications based on market feedback. Users enter bids to purchase quotas, or bundles of resources for long-term use. These requests are mapped into a simulated clock auction which determines uniform, fair resource prices that balance supply and demand. The reserve prices for resources sold by the operator in this auction are set based on current utilization, thus guiding the users as they set their bids towards under-utilized resources. By running these auctions at regular time intervals, prices fluctuate like those in a real-world economy and provide motivation for users to engineer systems that can best take advantage of available resources. These ideas were implemented in an experimental resource market at Google. Our preliminary results demon...

• Type: Conference
• Year: 2009
• Where: IPPS
• Authors: Murray Stokely, Jim Winget, E. Keyes, C. Grimes, Benjamin Yolken
• Keywords: Congested Resources, Distributed And Parallel Computing, IPPS 2009, Resource, Resource Provisioning Problem