The Basque task: did systems perform in the upperbound?

In this paper we describe the Senseval 2 Basque lexicalsample task. The task comprised 40 words (15 nouns, 15 verbs and 10 adjectives) selected from Euskal Hiztegia, the main Basque dictionary. Most examples were taken from the Egunkaria newspaper. The method used to handtag the examples produced low intertagger agreement (75%) before arbitration. The four competing systems attained results well above the most frequent baseline and the best system scored 75% precision at 100% coverage. The paper includes an analysis of the tagging procedure used, as well as the performance of the competing systems. In particular, we argue that intertagger agreement is not a real upperbound for the Basque WSD task.

• Type: Journal
• Year: 2002
• Where: CORR
• Authors: Eneko Agirre, Elena Garcia, Mikel Lersundi, David Martínez, Eli Pociello
• Keywords: Basque Lexicalsample Task, CORR 2002, Education, Intertagger Agreement, Low Intertagger Agreement