Sciweavers

30
Voted
JETAI
2010

Warning: statistical benchmarking is addictive. Kicking the habit in machine learning

13 years 9 months ago
Warning: statistical benchmarking is addictive. Kicking the habit in machine learning
Algorithm performance evaluation is so entrenched in the Machine Learning community that one could call it an addiction. Like most addictions, it is harmful and very difficult to give up. It is harmful because it has serious limitations. Yet, we have great faith in it practicing it in a ritualistic manner: we follow a fixed set of rules telling us the measure, the data sets and the statistical test to use. When we read a paper, even as reviewers, we are not sufficiently critical of results that follow these rules. Here, we will debate what are the limitations and how to best address them. This paper may not cure the addiction but hopefully it will be a good first step along that road.
Chris Drummond, Nathalie Japkowicz
Added 28 Jan 2011
Updated 28 Jan 2011
Type Journal
Year 2010
Where JETAI
Authors Chris Drummond, Nathalie Japkowicz
Comments (0)