Recommendations for Qualitative Ontology Matching Evaluations

This paper suggests appropriate rules to set up ontology matching evaluations and for golden standard construction and use which can significantly improve the quality of the precision and recall measures. We focus on the problem of evaluating ontology matching techniques [1] which find mappings with equivalence, less general, more general and disjointness, and on how to make the evaluation results fairer and more accurate. The literature discusses the appropriateness and quality of the measures [2], but contains little about evaluation methodology [3]. Closer to us, [4] raises the issue of evaluating non-equivalence links. Golden standards (GS) are fundamental for evaluating the precision and recall [2]. Typically, hand-made positive (GS+ ) and negative (GS− ) golden standards contain links considered correct and incorrect, respectively. Ideally, GS− complements GS+ , leading to a precise evaluation. Yet, in big datasets annotating all links is impractical and golden standards ar...

• Type: Conference
• Year: 2009
• Where: SEMWEB
• Authors: Aliaksandr Autayeu, Vincenzo Maltese, Pierre Andrews
• Keywords: Golden Standards, Internet Technology, Ontology Matching, Ontology Matching Evaluations, SEMWEB 2009