Exploring Correlated Subspaces for Efficient Query Processing in Sparse Databases

The sparse data is becoming increasingly common and available in many real-life applications. However, relative little attention has been paid to effectively model the sparse data and existing approaches such as the conventional "horizontal" and "vertical" representations fail to provide satisfactory performance for both storage and query processing, as such approaches are too rigid and generally do not consider the dimension correlations. In this paper, we propose a new approach, named HoVer, to store and conduct query for sparse datasets in an unmodified RDBMS, where HoVer stands for Horizontal representation over Vertically partitioned subspaces. According to the dimension correlations of sparse datasets, a novel mechanism has been developed to vertically partition a high-dimensional sparse dataset into multiple lower dimensional subspaces, and all the dimensions are highly correlated intrasubspace and highly unrelated inter-subspace respectively. Therefore, orig...

• Type: Journal
• Year: 2010
• Where: TKDE
• Authors: Bin Cui, Jiakui Zhao, Dongqing Yang
• Keywords: Education, Sparse, Sparse Data, Sparse Dataset, TKDE 2010