Simulation Study of Language Specific Web Crawling

The Web has been recognized as an important part of our cultural heritage. Many nations started archiving national web spaces for future generations. A key technology for data acquisition employed by these archiving projects is web crawling. Crawling cultural and/or linguistic specific resources from the borderless Web raises many challenging issues. In this paper, we investigate various approaches for language specific web crawling and evaluate them on the Web Crawling Simulator. Keyword Language Specific Web Crawling, Web Crawling Simulator, Web Archiving

• Type: Conference
• Year: 2005
• Where: ICDE
• Authors: Kulwadee Somboonviwat, Masaru Kitsuregawa, Takayuki Tamura
• Keywords: Database, ICDE 2005, Specific Web Crawling, Web Crawling, Web Crawling Simulator