Estimating the Change of Web Pages

This paper presents the estimation methods computing the probabilities of how many times web pages are downloaded and modified, respectively, in the future crawls. The methods can make web database administrators avoid unnecessarily requesting undownloadable and unmodified web pages in a page group. We postulate that the change behavior of web pages is strongly related to the past change behavior. We gather the change histories of approximately three million web pages at two-day intervals for 100 days, and estimate the future change behavior of those pages. Our estimation, which was evaluated by actual change behavior of the pages, worked well.

• Type: Conference
• Year: 2007
• Where: ICCS
• Authors: Sung Jin Kim, Sang Ho Lee
• Keywords: Applied Computing, Change Behavior, ICCS 2007, Times Web Pages, Web Pages