Consider a set of labels L and a set of trees T = {T (1) , T (2) , . . . , T (k) } where each tree T (i) is distinctly leaf-labeled by some subset of L. One fundamental problem is ...
Assessors frequently disagree on the topical relevance of documents. How much of this disagreement is due to ambiguity in assessment instructions? We have two assessors assess TRE...
We have analyzed system rankings for person name search algorithms using a data set for which several versions of ground truth were developed by employing different means of resol...
The paper presents a new fuzzy agreement measure f for determining the agreement in multi-label and subjective annotation task. In this annotation framework, one data item may bel...
Teamwork demands agreement among teammembers to collaborate and coordinate effectively. When a disagreement between teammates occurs (due to failures), team-members should ideally...