Do Clarity Scores for Queries Correlate with User Performance?

Recently the concept of a clarity score was introduced in order to measure the ambiguity of a query in relation to the collection in which the query issuer is seeking information [CronenTownsend et al. Proc. ACM SIGIR2002, Tampere Finland, August 2002]. If the query is expressed in the “same language” as the whole collection then it has a low clarity score, otherwise it has a high score, where the similarity is the relative entropy of the query and collection models. Cronen-Townsend et al. show that clarity scores correlate directly with average precision, hence a query with a high clarity score is likely to produce relevant documents high in a list of resulting documents. Other authors, however, have shown that high precision does not necessarily correlate with increased user performance. In this paper we examine the correlation between user performance and clarity score. Using log files from user experiments conducted within the framework of the TREC Interactive Track, we measu...

• Type: Conference
• Year: 2004
• Where: ADC
• Authors: Andrew Turpin, William R. Hersh
• Keywords: ADC 2004, Clarity Scores, Database, Low Clarity Score, User Performance