An adaptive and dynamic dimensionality reduction method for high-dimensional indexing

Abstract The notorious "dimensionality curse" is a wellknown phenomenon for any multi-dimensional indexes attempting to scale up to high dimensions. One well-known approach to overcome degradation in performance with respect to increasing dimensions is to reduce the dimensionality of the original dataset before constructing the index. However, identifying the correlation among the dimensions and effectively reducing them are challenging tasks. In this paper, we present an adaptive Multi-level Mahalanobisbased Dimensionality Reduction (MMDR) technique for high-dimensional indexing. Our MMDR technique has four notable features compared to existing methods. First, it discovers elliptical clusters for more effective dimensionality reduction by using only the low-dimensional subspaces. Second, data points in the different axis systems are indexed using a single B+-tree. Third, our technique is highly scalable in terms of data size and dimension. Finally, it is also dynamic and ada...

• Type: Conference
• Year: 2007
• Where: VLDB
• Authors: Heng Tao Shen, Xiaofang Zhou, Aoying Zhou
• Keywords: Database, Dimensionality Reduction, Effective Dimensionality Reduction, Mahalanobisbased Dimensionality Reduction, VLDB 2007