A Study on Evaluation on Opinion Retrieval Systems

We study the evaluation of opinion retrieval systems. Opinion retrieval is a relatively new research area, nevertheless classical evaluation measures, those adopted for ad hoc retrieval, such as MAP, precision at 10 etc., were used to assess the quality of rankings. In this paper we investigate the effectiveness of these standard evaluation measures for topical opinion retrieval. In doing this we split the opinion dimension from the relevance one and use opinion classifiers, with varying accuracy, to analyse how opinion retrieval performance changes by perturbing the outcomes of the opinion classifiers. Classifiers could be studied in two modalities, that is either to re-rank or to filter out directly documents obtained through a first relevance retrieval. In this paper we formally outline both approaches, while for now focussing on the filtering process. The proposed approach aims to establish the correlation between the accuracy of the classifiers and the performance of the topical ...

• Type: Conference
• Year: 2010
• Where: IIR
• Authors: Giambattista Amati, Giuseppe Amodeo, Valerio Capozio, Carlo Gaibisso, Giorgio Gambosi
• Keywords: IIR 2010, Information Technology, Opinion Retrieval, Opinion Retrieval Systems, Topical Opinion Retrieval