An Evaluation of the Accuracy of Capturing User Intent for Information Retrieval

Abstract This paper reports our evaluation of the accuracy of capturing a user’s intent in an informationseeking task. Specifically, we would like to assess how accurately a user’s short-term goals, methods, and context in an information seeking task have been captured. Our method is to compare a machine-generated model against a human-generated model with 5 users using the CACM collection. Our results demonstrate a good coverage of human-generated user models by machinegenerated models in terms of short-term goals, methods and context. The similarity between a real user and our approach in terms of context agrees with the similarities of human -generated ontologies using the same metrics in existing studies from the ontology community. Furthermore, our results show that the similarities between a human and machine in terms of short term goals and methods, which affect the relevancy assessment process, agree with the overlap among people while assessing relevancy documents in the...

• Type: Conference
• Year: 2007
• Where: ICAI
• Authors: Hien Nguyen, Eugene Santos Jr.
• Keywords: Artificial Intelligence, Human-generated User Models, ICAI 2007, Short-term Goals, User’s Short-term Goals