Mining Flipping Correlations from Large Datasets with Taxonomies

In this paper we introduce a new type of pattern – a flipping correlation pattern. The flipping patterns are obtained from contrasting the correlations between items at different levbstraction. They represent surprising correlations, both positive and negative, which are specific for a given abstraction level, and which “flip” from positive to negative and vice versa when items are generalized to a higher abstraction. We design an efficient algorithm for finding flipping correlations, the Flipper algorithm, which outperforms na¨ıve pattern mining methods by several orders of magnitude. We apply Flipper to real-life datasets and show that the discovered patterns are non-redundant, surprising and actionable. Flipper finds strong contrasting correlations in itemsets with low-to-medium support, while existing techniques cannot handle the pattern discovery in this frequency range. Categories and Subject Descriptors I.5.1 [Pattern Recognition]: Models—Statistical; H.2.8 [D...

• Type: Journal
• Year: 2012
• Where: CORR
• Authors: Marina Barsky, Sangkyum Kim, Tim Weninger, Jiawei Han
• Keywords: Abstraction Level, CORR 2012, Education, Pattern Discovery, Recognition Models