A non-linear dimensionality-reduction technique for fast similarity search in large databases

To enable efficient similarity search in large databases, many indexing techniques use a linear transformation scheme to reduce dimensions and allow fast approximation. In this reduction approach the approximation is unbounded, so that the approximation volume extends across the dataspace. This causes over-estimation of retrieval sets and impairs performance. This paper presents a non-linear transformation scheme that extracts two important parameters specifying the data. We prove that these parameters correspond to a bounded volume around the search sphere, irrespective of dimensionality. We use a special workspace-mapping mechanism to derive tight bounds for the parameters and to prove further results, as well as highlighting insights into the problems and our proposed solutions. We formulate a measure that lower-bounds the Euclidean distance, and discuss the implementation of the technique upon a popular index structure. Extensive experiments confirm the superiority of this techniq...

• Type: Conference
• Year: 2006
• Where: SIGMOD
• Authors: Khanh Vu, Kien A. Hua, Hao Cheng, Sheau-Dong Lang
• Keywords: Approximation Volume, Database, Fast Approximation, Non-linear Transformation Scheme, SIGMOD 2006