Exact model averaging with naive Bayesian classifiers

The naive classifier is a well-established mathematical model whose simplicity, speed and accuracy have made it a popular choice for classification in AI and engineering. In this paper we show that, given N features of interest, it is possible to perform tractable exact model averaging (MA) over all 2N possible feature-set models. In fact, we show that it is possible to calculate parameters for a single naive classifier C such that C produces predictions equivalent to those obtained by the full model-averaging, and we show that C can be constructed using the same time and space complexity required to construct a single naive classifier with MAP parameters. We present experimental results which show that on average the MA classifier typically outperforms the MAP classifier on simulated data, and we characterize how the relative performance varies with number of variables, number of training records, and complexity of the generating distribution. Finally, we examine the performance of t...

• Type: Conference
• Year: 2002
• Where: ICML
• Authors: Denver Dash, Gregory F. Cooper
• Keywords: ICML 2002, MA Naive Model, Machine Learning, MAP Classifier, Single Naive Classifier