From active towards InterActive learning: using consideration information to improve labeling correctness

Data mining techniques have become central to many applications. Most of those applications rely on so called supervised learning algorithms, which learn from given examples in the form of data with predefined labels (e.g., classes such as spam, not spam). Labeling, however, is oftentimes expensive, as it typically requires manual work by human experts. Active learning systems reduce the human effort by choosing the most informative instances for labeling. Unfortunately, research in psychology has shown conclusively that human decisions are inaccurate, easily biased by circumstances, and far from the oracle decision making assumed in active learning research. Based on these findings we show experimentally that (human) mistakes in labeling can significantly deteriorate the performance of active learning systems. To solve this problem, we introduce consideration information – a concept from marketing – into an active learning system to bias and improve the human’s labeling perform...

• Type: Conference
• Year: 2009
• Where: KDD
• Authors: Abraham Bernstein, Jiwen Li
• Keywords: Active Learning, Active Learning Systems, Consideration Information, Data Mining, KDD 2009