On PAC Learning Using Winnow, Perceptron, and a Perceptron-like Algorithm

In this paper we analyze the PAC learning abilities of several simple iterative algorithms for learning linear threshold functions, obtaining both positive and negative results. We show that Littlestone’s Winnow algorithm is not an efficient PAC learning algorithm for the class of positive linear threshold functions. We also prove that the Perceptron algorithm cannot efficiently learn the unrestricted class of linear threshold functions even under the uniform distribution on boolean examples. However, we show that the Perceptron algorithm can efficiently PAC learn the class of nested functions (a concept class known to be hard for Perceptron under arbitrary distributions) under the uniform distribution on boolean examples. Finally, we give a very simple Perceptron-like algorithm for learning origin-centered halfspaces under the uniform distribution on the unit sphere in ¡£¢¥¤ Unlike the Perceptron algorithm, which cannot learn in the presence of classification noise, the ne...

• Type: Conference
• Year: 1999
• Where: COLT
• Authors: Rocco A. Servedio
• Keywords: Algorithms, COLT 1999, Linear Threshold Functions, Machine Learning, Perceptron Algorithm