Towards parameter-free data mining

Most data mining algorithms require the setting of many input parameters. Two main dangers of working with parameter-laden algorithms are the following. First, incorrect settings may cause an algorithm to fail in finding the true patterns. Second, a perhaps more insidious problem is that the algorithm may report spurious patterns that do not really exist, or greatly overestimate the significance of the reported patterns. This is especially likely when the user fails to understand the role of parameters in the data mining process. Data mining algorithms should have as few parameters as possible, ideally none. A parameter-free algorithm would limit our ability to impose our prejudices, expectations, and presumptions on the problem at hand, and would let the data itself speak to us. In this work, we show that recent results in bioinformatics and computational theory hold great promise for a parameter-free datamining paradigm. The results are motivated by observations in Kolmogorov comple...

• Type: Conference
• Year: 2004
• Where: KDD
• Authors: Eamonn J. Keogh, Stefano Lonardi, Chotirat (Ann) Ratanamahatana
• Keywords: Data Mining, Data Mining Algorithms, KDD 2004, Most Data Mining, Off-the-shelf Compression Algorithm