Validation of a Dialog System for Language Learners

In this paper we present experiments related to the validation of spoken language understanding capabilities in a language and culture training system. In this application, word-level recognition rates are insufficient to characterize how well the system serves its users. We present the results of an annotation exercise that distinguishes instances of non-recognition due to learner error from instances due to poor system coverage. These statistics give a more accurate and interesting description of system performance, showing how the system could be improved without sacrificing the instructional value of rejecting learner utterances when they are poorly formed.

• Type: Journal
• Year: 2010
• Where: SIGDIAL
• Authors: Alicia Sagae, W. Lewis Johnson, Stephen Bodnar
• Keywords: Culture Training, Learner Error, Natural Language Processing, SIGDIAL 2010, Word-level Recognition Rates