This paper reports our experience of evaluating an application that uses visualization approaches to support instructors in Web based distance education. The evaluation took place in three stages: a focus group, an experimental study, and a semi-structured interview. In this paper we focus our attention on the focus group, and we will show how this evaluation approach can be very effective in uncovering unexpected problems that cannot be identified with analytic evaluations or controlled experiments. Categories and Subject Descriptors H.5.2 [Information interfaces and presentation] User Interfaces -Evaluation/methodology, Graphical user interfaces (GUI) General Terms Human Factors, Verification. Keywords Focus group, Human factors, Information visualization evaluation.