A Precise Metric for Measuring How Much Web Pages Change

A number of similarity metrics have been used to measure the degree of web page changes in the literature. When a web page changes, the metrics often represent the change differently. In this paper, we first define criteria for web page changes to evaluate the effectiveness of the metrics in terms of six important types of web page changes. Second, we propose a new similarity metric appropriate for measuring the degree of web page changes. Using real web pages and synthesized pages, we analyze the five existing metrics (i.e., the byte-wise comparison, the TF·IDF cosine distance, the word distance, the edit distance, and the shingling) and ours under the proposed criteria. The analysis result shows that our metric represents the changes more effectively than other metrics. We expect that our study can help users select an appropriate metric for particular web applications.

• Type: Conference
• Year: 2006
• Where: DASFAA
• Authors: Shin Young Kwon, Sang Ho Lee, Sung Jin Kim
• Keywords: DASFAA 2006, Database, Similarity Metric, Similarity Metric Appropriate, Web Page Changes