The Hub and Spoke Paradigm for CSR Evaluation

In this paper, we introduce the new paradigm used in the most recent ARPA-sponsoredContinuousSpeech Recognition(CSR) evaluation and then discuss the important features of the test design. The 1993 CSR evaluation was organized in a novel fashion in an attempt to accomodate research over a broad variety of important problems in CSR while maintaininga clear program-wide research focus. Furthermore, each test component in the evaluation was designed as an experiment to extract as much informationas possible from the results. The evaluation was centered around a large vocabulary speakerindependent (SI) baseline test, which was required of every participating site. This test was dubbed the 'Hub' since it was common to all sites and formed the basis for controlled inter-system comparisons. The Hub test was augmented with a variety of problem-specific optional tests designed to explore a variety of important problems in CSR, mostly involvingsome kind of mismatchbetween the training ...

• Type: Conference
• Year: 1994
• Where: NAACL
• Authors: Francis Kubala, Jerome R. Bellegarda, Jordan Cohen, David Pallett, Doug Paul, Mike Phillips, Raja Rajasekaran, Fred Richardson, Michael Riley, Roni Rosenfeld, Bob Roth, Mitch Weintraub
• Keywords: CSR Evaluation, Evaluation, Important Problems, NAACL 1994, NAACL 2007