Within the INitiative for the Evaluation of XML Retrieval (INEX) a number of metrics to evaluate the effectiveness of content-oriented XML retrieval approaches were developed. Although these metrics provide a solution towards addressing the problem of overlap among returned result elements, they do not consider the problem of overlapping reference components within the recall-base, hence leading to skewed effectiveness scores. We propose alternative metrics that aim to provide a solution to both overlap issues. Keywords XML retrieval, INEX, evaluation, metrics, overlap, overpopulated recall-base, cumulated gain
Gabriella Kazai, Mounia Lalmas, Arjen P. de Vries