Most existing performance evaluation methods concentrate on defining separate metrics over a wide range of conditions and generating standard benchmarking video sequences for examining the effectiveness of video tracking systems. In other words, these methods attempt to design a robustness margin or factor for the system. These methods are deterministic in which a robustness factor, for example, 2 or 3 times the expected number of subjects to track or the strength of illumination would be required in the design. This often results in overdesign, thus increasing costs, or underdesign causing failure by unanticipated factors. In order to overcome these limitations, we propose in this paper an alternative framework to analyze the physics of the failure process and, through the concept of reliability, determine the time to failure in automated video tracking systems. The benefit of our proposed framework is that we can provide a unified and statistical index to evaluate the performance of...