Formalism-Independent Parser Evaluation with CCG and DepBank

A key question facing the parsing community is how to compare parsers which use different grammar formalisms and produce different output. Evaluating a parser on the same resource used to create it can lead to non-comparable accuracy scores and an over-optimistic view of parser performance. In this paper we evaluate a CCG parser on DepBank, and demonstrate the difficulties in converting the parser output into DepBank grammatical relations. In addition we present a method for measuring the effectiveness of the conversion, which provides an upper bound on parsing accuracy. The

• Type: Conference
• Year: 2007
• Where: ACL
• Authors: Stephen Clark, James R. Curran
• Keywords: ACL 2007, Computational Linguistics, Non-comparable Accuracy Scores, Parser, Parser Performance