Constructing Ensembles from Data Envelopment Analysis

It has been shown in prior work in management science, statistics and machine learning that using an ensemble of models often results in better performance than using a single ‘best’ model. This paper proposes a novel Data Envelopment Analysis (DEA) based approach to combine models. We prove that for the 2-class classification problems, DEA models identify the same convex hull as the popular ROC analysis used for model combination. We further develop two DEA-based methods to combine k-class classifiers. Experiments demonstrate that the two methods outperform other benchmark methods and suggest that DEA can be a powerful tool for model combination. ______________________________ A preliminary version of this paper was accepted at the ACM Conference on Knowledge Discovery and Data Mining 2004 (KDD04). This version substantially extends the conference publication by a comprehensive literature review, a better model combination method, theoretical proofs and more experiments. 

• Type: Journal
• Year: 2007
• Where: INFORMS
• Authors: Zhiqiang Zheng, Balaji Padmanabhan
• Keywords: 2-class Classification Problems, INFORMS 2007, Model Combination, Popular Roc Analysis