Selecting actions for resource-bounded information extraction using reinforcement learning

Given a database with missing or uncertain content, our goal is to correct and fill the database by extracting specific information from a large corpus such as the Web, and to do so under resource limitations. We formulate the information gathering task as a series of choices among alternative, resource-consuming actions and use reinforcement learning to select the best action at each time step. We use temporal difference q-learning method to train the function that selects these actions, and compare it to an online, errordriven algorithm called SampleRank. We present a system that finds information such as email, job title and department affiliation for the faculty at our university, and show that the learning-based approach accomplishes this task efficiently under a limited action budget. Our evaluations show that we can obtain 92.4% of the final F1, by only using 14.3% of all possible actions. Categories and Subject Descriptors I.2.6 [Computing Methodologies]: Artificial Inte...

• Type: Journal
• Year: 2012
• Where: WSDM
• Authors: Pallika H. Kanani, Andrew K. McCallum
• Keywords: Data Mining, Information Acquisition, Q Learning, Resource Limitations, WSDM 2012