Sciweavers

ICML
2005
IEEE

Experimental comparison between bagging and Monte Carlo ensemble classification

14 years 12 months ago
Experimental comparison between bagging and Monte Carlo ensemble classification
Properties of ensemble classification can be studied using the framework of Monte Carlo stochastic algorithms. Within this framework it is also possible to define a new ensemble classifier, whose accuracy probability distribution can be computed exactly. This paper has two goals: first, an experimental comparison between the theoretical predictions and experimental results; second, a systematic comparison between bagging and Monte Carlo ensemble classification.
Roberto Esposito, Lorenza Saitta
Added 17 Nov 2009
Updated 17 Nov 2009
Type Conference
Year 2005
Where ICML
Authors Roberto Esposito, Lorenza Saitta
Comments (1)
me_village.jpgThe abstraction
[0]

I like the initial abstraction of this paper; namely that a distribution over hypotheses exists.

(Note: I suppose that in 2.2, second paragraph, it says that $p_j(x_k) \in \{1, 0\}$, this is a typo and it should be $[0,1]$.)

The theorem makes sense. However, it would be nice if it was made a bit more general. Right now it assumes that both bagging and MC share the same distribution over hypotheses.

Finally, the classical analysis of  a Bayesian classifier involves setting up a very similar construction. And, unless I am missing something, the standard MC methods applied to a Bayesian model would result in the same procedure.

In a Bayesian setting, with prior $p(w)$ over hypotheses, and $F_w(y|x)$ be an indicator function equal to 1 when hypothesis $w$ outputs $y$ for $x$, the algorithm would choose the label $y$ with maximum marginal probability
\[
P(y|x) = \int F_w(y|x) p(w) dw.
\]
One can sample from p(w) to obtain an MC estimate of P(y|x).

It would be nice to see a more thorough exploration of the relation of bagging to bayesian methods... are there any known results? I feel that the more general bootstrapping literature may have something.

C. Dimitrakakis

login or register to post comments