Efficient Heuristics for Discriminative Structure Learning of Bayesian Network Classifiers

We introduce a simple order-based greedy heuristic for learning discriminative structure within generative Bayesian network classifiers. We propose two methods for establishing an order of N features. They are based on the conditional mutual information and classification rate (i.e., risk), respectively. Given an ordering, we can find a discriminative structure with O Nk+1 score evaluations (where constant k is the tree-width of the sub-graph over the attributes). We present results on 25 data sets from the UCI repository, for phonetic classification using the TIMIT database, for a visual surface inspection task, and for two handwritten digit recognition tasks. We provide classification performance for both discriminative and generative parameter learning on both discriminatively and generatively structured networks. The discriminative structure found by our new procedures significantly outperforms generatively produced structures, and achieves a classification accuracy on par with th...

• Type: Journal
• Year: 2010
• Where: JMLR
• Authors: Franz Pernkopf, Jeff A. Bilmes
• Keywords: Bayesian Network Classifiers, Discriminative Classifiers, Discriminative Structure, JMLR 2010