Mining high-speed data streams

Many organizations today have more than very large databases; they have databases that grow without limit at a rate of several million records per day. Mining these continuous data streams brings unique opportunities, but also new challenges. This paper describes and evaluates VFDT, an anytime system that builds decision trees using constant memory and constant time per example. VFDT can incorporate tens of thousands of examples per second using off-the-shelf hardware. It uses Hoeffding bounds to guarantee that its output is asymptotically nearly identical to that of a conventional learner. We study VFDT's properties and demonstrate its utility through an extensive set of experiments on synthetic data. We apply VFDT to mining the continuous stream of Web access data from the whole University of Washington main campus. Categories and Subject Descriptors H.2.8 [Database Management]: Database Applications-data mining; I.2.6 [Artificial Intelligence]: Learning-concept learning; I.5.2...

• Type: Conference
• Year: 2000
• Where: KDD
• Authors: Pedro Domingos, Geoff Hulten
• Keywords: Continuous Data Streams, Data Mining, Database Applications-Data Mining, Decision Trees, KDD 2000