Features are often the basic unit of development for a very large software systems and represent long-term efforts, spanning up to several years from inception to actual use. Developing an experiment to monitor (by means of sampling) such lengthy processes requires a great deal of care in order to minimize costs and to maximize benefits. Just as prototyping is often a necessary auxiliary step in a large-scale, long-term development effort, so to is prototyping a necessary step in the development of a large-scale, long-term process monitoring experiment. Therefore, we have prototyped our experiment using a representative process and reconstructed data from a large and rich feature development. This prototyping approach has yielded two interesting sets of results. First, we found that we needed to modify our experimental design. Our initial set of states did not represent the data as well as we had hoped. This is exemplified by the fact that the ‘‘other’’ category is too large...
Mark G. Bradac, Dewayne E. Perry, Lawrence G. Vott