Average-Case Analysis of a Nearest Neighbor Algorithm

In this paper we present an average-case analysis of the nearest neighbor algorithm, a simple induction method that has been studied by manyresearchers. Our analysis assumes a conjunctive target concept, noise-free Boolean attributes, and a uniform distribution over the instance space. We calculate the probability that the algorithm will encounter a test instance that is distance d from the prototype of the concept, along with the probabilitythat the nearest stored training case is distance e from this test instance. From this we compute the probability of correct classi cation as a function of the number of observed training cases, the number of relevant attributes, and the number of irrelevant attributes. We also explore the behavioral implicationsof the analysis by presenting predicted learning curves for arti cial domains, and give experimental results on these domains as a check on our reasoning.

• Type: Conference
• Year: 1993
• Where: IJCAI
• Authors: Pat Langley, Wayne Iba
• Keywords: IJCAI 1993, IJCAI 2007, Nearest Neighbor Algorithm, Noise-free Boolean Attributes, Test Instance