One use of simulation is to inform decision makers that seek to select the best of several alternative systems. The system with the highest (or lowest) mean value for simulation output is often selected as best, and simulation output is used to infer the value of the unknown mean of each system. Statistical procedures that help to identify the best system by suggesting an appropriate number of replications for each system are therefore useful tools in simulation. This article explores the performance of representative procedures from two approaches to develop statistical procedures, with the goal of understanding tradeoffs involving the ease of use, computational requirements, and the range of applicability. The focus is primarily on procedures that use common random numbers to sharpen comparisons between systems.
Stephen E. Chick, Koichiro Inoue