Data Mining in the Real World: Experiences, Challenges, and Recommendations

Abstract - Data mining is used regularly in a variety of industries and is continuing to gain in both popularity and acceptance. However, applying data mining methods to complex real-world tasks is far from straightforward and many pitfalls face data mining practitioners. However, most research in the field tends to focus on the algorithmic issues that arise in data mining and ignores the human element and process issues that are often the cause of these pitfalls. While there are some papers on data mining experiences and lessons learned, they are quite rare, especially in the research community. The purpose of this paper is to begin to fill in the "gap" between data mining methods and the practice of data mining. This paper describes many of the experiences of the author as a data mining practitioner, highlights the issues that he encountered while in industry, and provides a number of strategies and recommendations for dealing with these issues. This paper should benefit bo...

• Type: Journal
• Year: 2009
• Where: DMIN
• Authors: Gary M. Weiss
• Keywords: Data Mining, Data Mining Methods, DMIN 2009, Mining