This paper presents a novel task-oriented approach for the evaluation of automatic text summarisation systems. Evaluation of systems has traditionally been a troublesome area in summarisation research. We propose a scheme that evaluates three existing systems by determining their relative effectiveness in an interactive search task, under conditions that approximate the intended use of the systems.
Paul McLellan, Anastasios Tombros, Joemon M. Jose,