An approach to benchmarking configuration complexity

Configuration is the process whereby components are assembled or adjusted to produce a functional system that operates at a specified level of performance. Today, the complexity of configuration is a major impediment to deploying and managing computer systems. We describe an approach to quantifying configuration complexity, with the ultimate goal of producing a configuration complexity benchmark. Our belief is that such a benchmark can drive progress towards self-configuring systems. Unlike traditional workload-based performance benchmarks, our approach is process-based. It generates metrics that reflect the level of human involvement in the configuration process, quantified by interaction time and probability of successful configuration. It computes the metrics using a model of a standardized human operator, calibrated in advance by a user study that measures operator behavior on a set of parameterized canonical configuration actions. The model captures the human component of configu...

• Type: Conference
• Year: 2004
• Where: SIGOPSE
• Authors: Aaron B. Brown, Joseph L. Hellerstein
• Keywords: Configuration Complexity, Configuration Complexity Benchmark, Operating Systems, Parameterized Canonical Configuration, SIGOPSE 2004