Discovering rare categories and classifying new instances of them is
an important data mining issue in many fields, but fully supervised
learning of a rare class classifier is prohibitively costly. There has
therefore been increasing interest both in active discovery: to
identify new classes quickly, and active learning: to train
classifiers with minimal supervision. Very few studies have attempted
to jointly solve these two inter-related tasks which occur together in
practice. Optimizing both rare class discovery and classification
simultaneously with active learning is challenging because discovery
and classification have conflicting requirements in query criteria. In
this paper we address these issues with two contributions: a unified
active learning model to jointly discover new categories and learn to
classify them; and a classifier combination algorithm that switches
generative and discriminative classifiers as learning
progresses. Extensive evaluation on several st...