Horizontal Aggregations in SQL to Prepare Data Sets for Data Mining Analysis

—Preparing a data set for analysis is generally the most time consuming task in a data mining project, requiring many complex SQL queries, joining tables and aggregating columns. Existing SQL aggregations have limitations to prepare data sets because they return one column per aggregated group. In general, a significant manual effort is required to build data sets, where a horizontal layout is required. We propose simple, yet powerful, methods to generate SQL code to return aggregated columns in a horizontal tabular layout, returning a set of numbers instead of one number per row. This new class of functions is called horizontal aggregations. Horizontal aggregations build data sets with a horizontal denormalized layout (e.g. point-dimension, observation-variable, instance-feature), which is the standard layout required by most data mining algorithms. We propose three fundamental methods to evaluate horizontal aggregations: CASE: Exploiting the programming CASE construct; SPJ: Based ...

• Type: Journal
• Year: 2012
• Where: TKDE
• Authors: Carlos Ordonez, Zhibo Chen 0002
• Keywords: Data Mining Project, Formal Methods, Linear Scalability, Tabular Layout, TKDE 2012