Improving Random Forests

Random forests are one of the most successful ensemble methods which exhibits performance on the level of boosting and support vector machines. The method is fast, robust to noise, does not overfit and offers possibilities for explanation and visualization of its output. We investigate some possibilities to increase strength or decrease correlation of individual trees in the forest. Using several attribute evaluation measures instead of just one gives promising results. On the other hand replacement of ordinary voting with voting weighted with margin achieved on most similar instances gives improvements which are statistically highly significant over several data sets.

• Type: Conference
• Year: 2004
• Where: ECML
• Authors: Marko Robnik-Sikonja
• Keywords: ECML 2004, Instances Gives Improvements, Successful Ensemble Methods, Support Vector Machines