A Critical Reassessment of Evaluation Baselines for Speech Summarization

We assess the current state of the art in speech summarization, by comparing a typical summarizer on two different domains: lecture data and the SWITCHBOARD corpus. Our results cast significant doubt on the merits of this area's accepted evaluation standards in terms of: baselines chosen, the correspondence of results to our intuition of what "summaries" should be, and the value of adding speechrelated features to summarizers that already use transcripts from automatic speech recognition (ASR) systems. 1 Problem definition and related literature Speech is arguably the most basic, most natural form of human communication. The consistent demand for and increasing availability of spoken audio content on web pages and other digital media should therefore come as no surprise. Along with this availability comes a demand for ways to better navigate through speech, which is inherently more linear or sequential than text in its traditional delivery. Navigation connotes a number ...

• Type: Conference
• Year: 2008
• Where: ACL
• Authors: Gerald Penn, Xiaodan Zhu
• Keywords: ACL 2008, Automatic Speech Recognition, Computational Linguistics, Related Literature Speech, Speech Summarization