Evaluating Predictive Uncertainty Challenge

This Chapter presents the PASCAL1 Evaluating Predictive Uncertainty Challenge, introduces the contributed Chapters by the participants who obtained outstanding results, and provides a discussion with some lessons to be learnt. The Challenge was set up to evaluate the ability of Machine Learning algorithms to provide good “probabilistic predictions”, rather than just the usual “point predictions” with no measure of uncertainty, in regression and classification problems. Participants had to compete on a number of regression and classification tasks, and were evaluated by both traditional losses that only take into account point predictions and losses we proposed that evaluate the quality of the probabilistic predictions. 1 Motivation Information about the uncertainty of predictions, or predictive uncertainty, is essential in decision making. Aware of the traumatic cost of an operation, a surgeon will only decide to operate if there is enough evidence of cancer in the diagnostic...

• Type: Conference
• Year: 2005
• Where: MLCW
• Authors: Joaquin Quiñonero Candela, Carl Edward Rasmussen, Fabian H. Sinz, Olivier Bousquet, Bernhard Schölkopf
• Keywords: Machine Learning, MLCW 2005, Point Prediction, Predictive Uncertainty, Probabilistic Predictions