229 search results - page 7 / 46 » Evaluation measures for preference judgments |
CIKM
2003
Springer
14 years 1 months ago
2003
Springer
Evaluation of IR systems has always been difficult because of the need for manually assessed relevance judgments. The advent of large editor-driven taxonomies on the web opens the...
BMCBI
2008
13 years 8 months ago
2008
Background: The evaluation of information retrieval techniques has traditionally relied on human judges to determine which documents are relevant to a query and which are not. Thi...
WSDM
2010
ACM
14 years 5 months ago
2010
ACM
While test collection construction is a time-consuming and expensive process, the true cost is amortized by reusing the collection over hundreds or thousands of experiments. Some ...
NAACL
2010
13 years 6 months ago
2010
Translation systems are generally trained to optimize BLEU, but many alternative metrics are available. We explore how optimizing toward various automatic evaluation metrics (BLEU...
WWW
2009
ACM
14 years 9 months ago
2009
ACM
We consider the problem of identifying the consensus ranking for the results of a query, given preferences among those results from a set of individual users. Once consensus ranki...