Despite recent advances in natural language question answering technology, the problem of designing effective user interfaces has been largely unexplored. We conducted a user study to investigate the problem and discovered that overall, users prefer a paragraph-sized chunk of text over just an exact phrase as the answer to their questions. Furthermore, users generally prefer answers embedded in context, regardless of the perceived reliability of the source documents. When users research a topic, increasing the amount of text returned to users significantly decreases the number of queries that they pose to the system, suggesting that users utilize supporting text to answer related questions. We believe that these results can serve to guide future developments in question answering user interfaces. Keywords Question answering, information retrieval, user interface, natural language
Jimmy J. Lin, Dennis Quan, Vineet Sinha, Karun Bak