Adaptive Cluster Distance Bounding for High-Dimensional Indexing

—We consider approaches for similarity search in correlated, high-dimensional data-sets, which are derived within a clustering framework. We note that indexing by “vector approximation” (VA-File), which was proposed as a technique to combat the “Curse of Dimensionality”, employs scalar quantization, and hence necessarily ignores dependencies across dimensions, which represents a source of suboptimality. Clustering, on the other hand, exploits inter-dimensional correlations and is thus a more compact representation of the data-set. However, existing methods to prune irrelevant clusters are based on bounding hyperspheres and/or bounding rectangles, whose lack of tightness compromises their efficiency in exact nearest neighbor search. We propose a new cluster-adaptive distance bound based on separating hyperplane boundaries of Voronoi clusters to complement our cluster based index. This bound enables efficient spatial filtering, with a relatively small pre-processing storage ...

• Type: Journal
• Year: 2011
• Where: TKDE
• Authors: Sharadh Ramaswamy, Kenneth Rose
• Keywords: Exact Nearest Neighbor, Sequential Io Operations, Small Pre-processing Storage, TKDE 2011