On-line discovery of temporal-difference networks

We present an algorithm for on-line, incremental discovery of temporal-difference (TD) networks. The key contribution is the establishment of three criteria to expand a node in TD network: a node is expanded when the node is well-known, independent, and has a prediction error that requires further explanation. Since none of these criteria requires centralized calculation operations, they are easily computed in a parallel and distributed manner, and scalable for bigger problems compared to other discovery methods of predictive state representations. Through computer experiments, we demonstrate the empirical effectiveness of our algorithm.

• Type: Conference
• Year: 2008
• Where: ICML
• Authors: Takaki Makino, Toshihisa Takagi
• Keywords: Centralized Calculation Operations, ICML 2008, Machine Learning, Predictive State Representations, TD Network