Cutting Corners: Workbench Automation for Server Benchmarking

A common approach to benchmarking a server is to measure its behavior under load from a workload generator. Often a set of such experiments is required-perhaps with different server configurations or workload parameters--to obtain a statistically sound result for a given benchmarking objective. This paper explores a framework and policies to conduct such benchmarking activities automatically and efficiently. The workbench automation framework is designed to be independent of the underlying benchmark harness, including the server implementation, configuration tools, and workload generator. Rather, we take those mechanisms as given and focus on automation policies within the framework. As a motivating example we focus on rating the peak load of an NFS file server for a given set of workload parameters, a common and costly activity in the storage server industry. Experimental results show how an automated workbench controller can plan and coordinate the benchmark runs to obtain a result ...

• Type: Conference
• Year: 2008
• Where: USENIX
• Authors: Piyush Shivam, Varun Marupadi, Jeffrey S. Chase, Thileepan Subramaniam, Shivnath Babu
• Keywords: Benchmarking, Operating System, USENIX 2008, Workload, Workload Generator