Inferring document similarity from hyperlinks

Assessing semantic similarity between text documents is a crucial aspect in Information Retrieval systems. In this work, we propose to use hyperlink information to derive a similarity measure that can then be applied to compare any text documents, with or without hyperlinks. As linked documents are generally semantically closer than unlinked documents, we use a training corpus with hyperlinks to infer a function a, b → sim(a, b) that assigns a higher value to linked documents than to unlinked ones. Two sets of experiments on different corpora show that this function compares favorably with OKAPI matching on document retrieval tasks. Categories and Subject Descriptors: H.3.3 [Information Storage and Retrieval]: Miscellaneous I.2.6 [Artificial Intelligence]: Learning General Terms: Algorithms, Experimentation

• Type: Conference
• Year: 2005
• Where: CIKM
• Authors: David Grangier, Samy Bengio
• Keywords: CIKM 2005, Information Retrieval Systems, Semantic Similarity, Text Documents