Visual Terrain Analysis of High-Dimensional Datasets

Abstract. Most real-world datasets are, to a certain degree, skewed. When considered that they are also large, they become the pinnacle challenge in data analysis. More importantly, we cannot ignore such datasets as they arise frequently in a wide variety of applications. Regardless of the analytic, it is often that the effectiveness of analysis can be improved if the characteristic of the dataset is known in advance. In this paper, we propose a novel technique to preprocess such datasets to obtain this insight. Our work is inspired by the resonance phenomenon, where similar objects resonate to a given response function. The key analytic result of our work is the data terrain, which shows properties of the dataset to enable effective and efficient analysis. We demonstrated our work in the context of various real-world problems. In doing so, we establish it as the tool for preprocessing data before applying computationally expensive algorithms.

• Type: Conference
• Year: 2005
• Where: PKDD
• Authors: Wenyuan Li, Kok-Leong Ong, Wee Keong Ng
• Keywords: Certain Degree, Data Analysis, Most Real-world Datasets, PKDD 2005