Single-pass online learning: performance, voting schemes and online feature selection

To learn concepts over massive data streams, it is essential to design inference and learning methods that operate in real time with limited memory. Online learning methods such as perceptron or Winnow are naturally suited to stream processing; however, in practice multiple passes over the same training data are required to achieve accuracy comparable to state-of-the-art batch learners. In the current work we address the problem of training an on-line learner with a single pass over the data. We evaluate several existing methods, and also propose a new modification of Margin Balanced Winnow, which has performance comparable to linear SVM. We also explore the effect of averaging, a.k.a. voting, on online learning. Finally, we describe how the new Modified Margin Balanced Winnow algorithm can be naturally adapted to perform feature selection. This scheme performs comparably to widely-used batch feature selection methods like information gain or Chi-square, with the advantage of being ab...

• Type: Conference
• Year: 2006
• Where: KDD
• Authors: Vitor R. Carvalho, William W. Cohen
• Keywords: Data Mining, KDD 2006, Margin Balanced Winnow, Online Learning Methods, Winnow Algorithm