When Does Greedy Learning of Relevant Features Succeed? --- A Fourier-based Characterization ---

Detecting the relevant attributes of an unknown target concept is an important and well studied problem in algorithmic learning. Simple greedy strategies have been proposed that seem to perform reasonably well in practice if a sufficiently large random subset of examples of the target concept is provided. Introducing a new notion called Fourier-accessibility allows us to characterize the class of Boolean functions precisely for which a standard greedy learning algorithm successfully learns all relevant attributes. Technically, this is achieved by deriving new relations between the learnability of a function and its Fourier spectrum. We prove that if the target concept is Fourier-accessible, then the success probability of the greedy algorithm can be made arbitrarily close to one. On the other hand, if the target concept is not Fourier-accessible, then the error probability tends to one.

• Type: Journal
• Year: 2006
• Where: ECCC
• Authors: Jan Arpe, Rüdiger Reischuk
• Keywords: ECCC 2006, Relevant Attributes, Target Concept, Unknown Target Concept