Using observational pilot studies to test and improve lab packages

Controlled experiments are a key approach to evaluate and evolve our understanding of software engineering technologies. However, defining and running a controlled experiment is a difficult and error-prone task. This paper argues that one can significantly reduce the risks associated with defining a new controlled experiment by running a set of well-planned observational pilot studies aimed at improving the experimental material. It gives the steps for such approach and uses a case study to illustrate it. The case study shows the definition of an experiment to evaluate inspection techniques for information visualization tools through a set of four observational studies and one experimental trial. Based on the lessons learned we present some guidelines on how to test and improve experimental material in such way. Categories and Subject Descriptors D.2.0 [Software Engineering]: General General Terms Measurement, Experimentation. Keywords Empirical Studies, Lab Package, Visual Data Minin...

• Type: Conference
• Year: 2006
• Where: ISESE
• Authors: Manoel G. Mendonça, Daniela Cruzes, Josemeire Dias, Maria Cristina Ferreira de Oliveira
• Keywords: Case Study, ISESE 2006, Software Engineering, Well-planned Observational Pilot