Representation Discovery in Sequential Decision Making

Automatically constructing novel representations of tasks from analysis of state spaces is a longstanding fundamental challenge in AI. I review recent progress on this problem for sequential decision making tasks modeled as Markov decision processes. Specifically, I discuss three classes of representation discovery problems: finding functional, state, and abstractions. I describe solution techniques varying along several dimensions: diagonalization or dilation methods using approximate or exact transition models; rewardspecific vs reward-invariant methods; global vs. local representation construction methods; multiscale vs. flat discovery methods; and finally, orthogonal vs. redundant representation discovery methods. I conclude by describing a number of open problems for future work.

• Type: Conference
• Year: 2010
• Where: AAAI
• Authors: Sridhar Mahadevan
• Keywords: AAAI 2010, Discovery Methods, Intelligent Agents, Representation Discovery, Vs Reward-invariant Methods