Tell me something I don't know: randomization strategies for iterative data mining

There is a wide variety of data mining methods available, and it is generally useful in exploratory data analysis to use many different methods for the same dataset. This, however, leads to the problem of whether the results found by one method are a reflection of the phenomenon shown by the results of another method, or whether the results depict in some sense unrelated properties of the data. For example, using clustering can give indication of a clear cluster structure, and computing correlations between variables can show that there are many significant correlations in the data. However, it can be the case that the correlations are actually determined by the cluster structure. In this paper, we consider the problem of randomizing data so that previously discovered patterns or models are taken into account. The randomization methods can be used in iterative data mining. At each step in the data mining process, the randomization produces random samples from the set of data matrices ...

• Type: Conference
• Year: 2009
• Where: KDD
• Authors: Heikki Mannila, Kai Puolamäki, Markus Ojala, Niko Vuokko, Nikolaj Tatti, Sami Hanhijärvi
• Keywords: Data Mining, Data Mining Methods, Data Mining Process, Iterative Data Mining, KDD 2009