Benchmarking natural-language parsers for biological applications using dependency graphs

Background: Interest is growing in the application of syntactic parsers to natural language processing problems in biology, but assessing their performance is difficult because differences in linguistic convention can falsely appear to be errors. We present a method for evaluating their accuracy using an intermediate representation based on dependency graphs, in which the semantic relationships important in most information extraction tasks are closer to the surface. We also demonstrate how this method can be easily tailored to various application-driven criteria. Results: Using the GENIA corpus as a gold standard, we tested four open-source parsers which have been used in bioinformatics projects. We first present overall performance measures, and test the two leading tools, the Charniak-Lease and Bikel parsers, on subtasks tailored to reflect the requirements of a system for extracting gene expression relationships. These two tools clearly outperform the other parsers in the evaluati...

• Type: Journal
• Year: 2007
• Where: BMCBI
• Authors: Andrew B. Clegg, Adrian J. Shepherd
• Keywords: BMCBI 2007, Dependency Graphs, Natural Language, Parsers