Visualizing differences in web search algorithms using the expected weighted hoeffding distance

We introduce a new dissimilarity function for ranked lists, the expected weighted Hoeffding distance, that has several advantages over current dissimilarity measures for ranked search results. First, it is easily customized for users who pay varying degrees of attention to websites at different ranks. Second, unlike existing measures such as generalized Kendall’s tau, it is based on a true metric, preserving meaningful embeddings when visualization techniques like multi-dimensional scaling are applied. Third, our measure can effectively handle partial or missing rank information while retaining a probabilistic interpretation. Finally, the measure can be made computationally tractable and we give a highly efficient algorithm for computing it. We then apply our new metric with multi-dimensional scaling to visualize and explore relationships between the result sets from different search engines, showing how the weighted Hoeffding distance can distinguish important differences in s...

• Type: Conference
• Year: 2010
• Where: WWW
• Authors: Mingxuan Sun, Guy Lebanon, Kevyn Collins-Thompson
• Keywords: Internet Technology, Multi-Dimensional Scaling, Search Engines, Weighted Hoeffding Distance, WWW 2010