A Unified Approach to Active Dual Supervision for Labeling Features and Examples

Abstract. When faced with the task of building accurate classifiers, active learning is often a beneficial tool for minimizing the requisite costs of human annotation. Traditional active learning schemes query a human for labels on intelligently chosen examples. However, human effort can also be expended in collecting alternative forms of annotation. For example, one may attempt to learn a text classifier by labeling words associated with a class, instead of, or in addition to, documents. Learning from two different kinds of supervision adds a challenging dimension to the problem of active learning. In this paper, we present a unified approach to such active dual supervision: determining which feature or example a classifier is most likely to benefit from having labeled. Empirical results confirm that appropriately querying for both example and feature labels significantly reduces overall human effort--beyond what is possible through traditional one-dimensional active learning.

• Type: Journal
• Year: 2010
• Where: PKDD
• Authors: Josh Attenberg, Prem Melville, Foster J. Provost
• Keywords: Active Learning, Data Mining, Human, Human Effort, PKDD 2010