Variability in Architectural Simulations of Multi-Threaded Workloads

Multi-threaded commercial workloads implement many important internet services. Consequently, these workloads are increasingly used to evaluate the performance of uniprocessor and multiprocessor system designs. This paper identifies performance variability as a potentially major challenge for architectural simulation studies using these workloads. Variability refers to the differences between multiple estimates of a workload's performance. Time variability occurs when a workload exhibits different characteristics during different phases of a single run. Space variability occurs when small variations in timing cause runs starting from the same initial condition to follow widely different execution paths. Variability is a well-known phenomenon in real systems, but is nearly universally ignored in simulation experiments. In a central result of this paper, we show that variability in multi-threaded commercial workloads can lead to incorrect architectural conclusions (e.g., 31% of the...

• Type: Conference
• Year: 2003
• Where: HPCA
• Authors: Alaa R. Alameldeen, David A. Wood
• Keywords: Computer Architecture, HPCA 2003, Multi-threaded Commercial Workloads, Performance Variability, Time Variability