An empirical evaluation of supervised learning in high dimensions

In this paper we perform an empirical evaluation of supervised learning on highdimensional data. We evaluate performance on three metrics: accuracy, AUC, and squared loss and study the effect of increasing dimensionality on the performance of the learning algorithms. Our findings are consistent with previous studies for problems of relatively low dimension, but suggest that as dimensionality increases the relative performance of the learning algorithms changes. To our surprise, the method that performs consistently well across all dimensions is random forests, followed by neural nets, boosted trees, and SVMs.

• Type: Conference
• Year: 2008
• Where: ICML
• Authors: Rich Caruana, Nikolaos Karampatziakis, Ainur Yessenalina
• Keywords: Algorithms Changes, ICML 2008, Machine Learning, Relative Performance, Supervised Learning