How is the weather tomorrow?: towards a benchmark for the cloud

Traditionally, the goal of benchmarking a software system is to evaluate its performance under a particular workload for a fixed configuration. The most prominent examples for evaluating transactional database systems as well as other components on top (such as a application-servers or web-servers) are the various TPC benchmarks. In this paper we argue that traditional benchmarks (like the TPC benchmarks) are not sufficient for analyzing the novel cloud services. Moreover, we present some initial ideas how such a new benchmark should look like that fits better to the characteristics of cloud computing (e.g., scalability, pay-per-use and fault-tolerance). The main challenge of such a new benchmark is to make the reported results comparable because different providers offer different services with different capabilities and guarantees.

• Type: Conference
• Year: 2009
• Where: SIGMOD
• Authors: Carsten Binnig, Donald Kossmann, Tim Kraska, Simon Loesing
• Keywords: Database, SIGMOD 2009, Traditional Benchmarks, Transactional Database Systems, Various Tpc Benchmarks