Query by Committee

We propose an algorithm called query by committee, in which a committee of students is trained on the same data set. The next query is chosen according to the principle of maximal disagreement. The algorithmis studied for two toy models: the high-low game and perceptron learning ofanother perceptron. As the number of queries goes to in nity, the committee algorithm yields asymptotically nite information gain. This leads to generalization error that decreases exponentially with the numberof examples. This in marked contrast to learning fromrandomlychosen inputs, for which the information gain approaches zero and the generalization error decreases with a relatively slow inverse power law. We suggest that asymptotically nite informationgain maybe an important characteristic of good query algorithms.

• Type: Conference
• Year: 1992
• Where: COLT
• Authors: H. Sebastian Seung, Manfred Opper, Haim Sompolinsky
• Keywords: Algorithm Called Query, COLT 1992, Generalization Error, Information Gain, Machine Learning