Sciweavers

33 search results - page 3 / 7
» Collecting Gold
Sort
View
EMNLP
2008
13 years 9 months ago
Cheap and Fast - But is it Good? Evaluating Non-Expert Annotations for Natural Language Tasks
Human linguistic annotation is crucial for many natural language processing tasks but can be expensive and time-consuming. We explore the use of Amazon's Mechanical Turk syst...
Rion Snow, Brendan O'Connor, Daniel Jurafsky, Andr...
SIGIR
2008
ACM
13 years 7 months ago
Relevance assessment: are judges exchangeable and does it matter
We investigate to what extent people making relevance judgements for a reusable IR test collection are exchangeable. We consider three classes of judge: "gold standard" ...
Peter Bailey, Nick Craswell, Ian Soboroff, Paul Th...
ERCIMDL
2009
Springer
153views Education» more  ERCIMDL 2009»
14 years 2 months ago
SyGAR - A Synthetic Data Generator for Evaluating Name Disambiguation Methods
Name ambiguity in the context of bibliographic citations is one of the hardest problems currently faced by the digital library community. Several methods have been proposed in the ...
Anderson A. Ferreira, Marcos André Gon&cced...
ACL
2008
13 years 9 months ago
In-Browser Summarisation: Generating Elaborative Summaries Biased Towards the Reading Context
We investigate elaborative summarisation, where the aim is to identify supplementary information that expands upon a key fact. We envisage such summaries being useful when browsin...
Stephen Wan, Cécile Paris
WCE
2007
13 years 8 months ago
Qualitative and Quantitative Criteria for the Concept Evaluation Task
act—Ontological concept evaluation is a difficult task. Till now, it is done either by domain expert or a knowledge base (thesaurus, ontology, etc.). In this research, we propose...
Lobna Karoui, Supelec France, Nabil El-Kadhi