Towards an Objective Test for Meeting Browsers: The BET4TQB Pilot Experiment

This paper outlines first the BET method for task-based evaluation of meeting browsers. ‘Observations of interest’ in meetings are empirically determined by neutral observers and then processed and ordered by evaluators. The evaluation of the TQB annotation-driven meeting browser using the BET is then described. A series of subjects attempted to answer as many meeting-related questions as possible in a fixed amount of time, and their performance was measured in terms of precision and speed. The results indicate that the TQB interface is easy to understand with little prior learning and that its annotation-based search functionality is highly relevant, in particular keyword search over the meeting transcript. Two knowledge-poorer browsers appear to offer lower precision but higher speed. The BET task-based evaluation method thus appears to be a coherent measure of browser quality. Key words: multimedia meeting browsers, task-based evaluation, humancomputer interaction, human fact...

• Type: Conference
• Year: 2007
• Where: MLMI
• Authors: Andrei Popescu-Belis, Philippe Baudrion, Mike Flynn, Pierre Wellner
• Keywords: BET Task-based Evaluation, Machine Learning, Meeting Browsers, MLMI 2007, Task-based Evaluation