Combining Text and Link Analysis for Focused Crawling

The number of vertical search engines and portals has rapidly increased over the last years, making the importance of a topic-driven (focused) crawler evident. In this paper, we develop a latent semantic indexing classifier that combines link analysis with text content in order to retrieve and index domain specific web documents. We compare its efficiency with other well-known web information retrieval techniques. Our implementation presents a different approach to focused crawling and aims to overcome the limitations of the neccesity to provide initial training data while maintaining a high recall/precision ratio.

• Type: Conference
• Year: 2005
• Where: ICAPR
• Authors: George Almpanidis, Constantine Kotropoulos
• Keywords: ICAPR 2005, Index Domain Specific, Latent Semantic Indexing, Vertical Search Engines