Regret Bounds for Sleeping Experts and Bandits

We study on-line decision problems where the set of actions that are available to the decision algorithm vary over time. With a few notable exceptions, such problems remained largely unaddressed in the literature, despite their applicability to a large number of practical problems. Departing from previous work on this "Sleeping Experts" problem, we compare algorithms against the payoff obtained by the best ordering of the actions, which is a natural benchmark for this type of problem. We study both the full-information (best expert) and partial-information (multi-armed bandit) settings and consider both stochastic and adaptive adversaries. For all settings we give algorithms achieving (almost) information-theoretically optimal regret bounds (up to a constant or a sublogarithmic factor) with respect to the best-ordering benchmark.

• Type: Conference
• Year: 2008
• Where: COLT
• Authors: Robert D. Kleinberg, Alexandru Niculescu-Mizil, Yogeshwer Sharma
• Keywords: Algorithms, COLT 2008, Decision Algorithm, Machine Learning, On-line Decision Problems