Learning physically-instantiated game play through visual observation

Abstract— We present an integrated vision and robotic system that plays, and learns to play, simple physically-instantiated board games that are variants of TIC TAC TOE and HEXAPAWN. We employ novel custom vision and robotic hardware designed specifically for this learning task. The game rules can be parametrically specified. Two independent computational agents alternate playing the two opponents with the shared vision and robotic hardware, using pre-specified rule sets. A third independent computational agent, sharing the same hardware, learns the game rules solely by observing the physical play, without access to the pre-specified rule set, using inductive logic programming with minimal background knowledge possessed by human children. The vision component of our integrated system reliably detects the position of the board in the image and reconstructs the game state after every move, from a single image. The robotic component reliably moves pieces both between board positions...

• Type: Journal
• Year: 2010
• Where: ICRA
• Authors: Andrei Barbu, Siddharth Narayanaswamy, Jeffrey Mark Siskind
• Keywords: ICRA 2010, Independent Computational Agent, Physical Play, Robotic, Robotics