Detecting soft errors by redirection classification

A soft error redirection is a URL redirection to a page that returns the HTTP status code 200 (OK) but has actually no relevant content to the client request. Since such redirections degrade the performance of web search engines in many ways, it is highly desirable to remove as many of them as possible. We propose a novel approach to detect soft error redirections by analyzing redirection logs collected during crawling operation. Experimental results on huge crawl data show that our measure can classify soft error redirections effectively. Categories and Subject Descriptors: H.3.3 [Information Systems]: Information Search and Retrieval; H.3.5 [Information Systems]: Online Information Services?Web based services General Terms: Experimentation, Measurement

• Type: Conference
• Year: 2009
• Where: WWW
• Authors: Taehyung Lee, Jinil Kim, Jin Wook Kim, Sung-Ryul Kim, Kunsoo Park
• Keywords: Internet Technology, Redirection Logs, Soft Error Redirections, URL Redirection, WWW 2009