Combining expert advice in reactive environments

"Experts algorithms" constitute a methodology for choosing actions repeatedly, when the rewards depend both on the choice of action and on the unknown current state of the environment. An experts algorithm has access to a set of strategies ("experts"), each of which may recommend which action to choose. The algorithm learns how to combine the recommendations of individual experts so that, in the long run, for any fixed sequence of states of the environment, it does as well as the best expert would have done relative to the same sequence. This methodology may not be suitable for situations where the evolution of states of the environment depends on past chosen actions, as is usually the case, for example, in a repeated non-zero-sum game. A general exploration-exploitation experts method is presented along with a proper definition of value. The new method is quite different from previously proposed experts algorithms. It represents a shift from the paradigms of regre...

• Type: Journal
• Year: 2006
• Where: JACM
• Authors: Daniela Pucci de Farias, Nimrod Megiddo
• Keywords: Algorithms, Expert, Experts Algorithms, JACM 2006