Learning when to stop thinking and do something!

An anytime algorithm is capable of returning a response to the given task at essentially any time; typically the quality of the response improves as the time increases. Here, we consider the challenge of learning when we should terminate such algorithms on each of a sequence of iid tasks, to optimize the expected average reward per unit time. We provide a system for addressing this challenge, which combines the global optimizer CrossEntropy method with local gradient ascent. This paper theoretically investigates how far the estimated gradient is from the true gradient, then empirically demonstrates that this system is effective by applying it to a toy problem, as well as on a real-world face detection task.

• Type: Conference
• Year: 2009
• Where: ICML
• Authors: Barnabás Póczos, Csaba Szepesvári, Nathan R. Sturtevant, Russell Greiner, Yasin Abbasi-Yadkori
• Keywords: Expected Average Reward, Face Detection Task, ICML 2009, Local Gradient Ascent, Machine Learning