Audio Feature Extraction and Analysis for Scene Segmentation and Classification

Understanding of the scene content of a video sequence is very important for content-based indexing and retrieval of multimedia databases. Research in this area in the past several years has focused on the use of speech recognition and image analysis techniques. As a complimentary effort to the prior work, we have focused on using the associated audio information (mainly the nonspeech portion) for video scene analysis. As an example, we consider the problem of discriminating five types of TV programs, namely commercials, basketball games, football games, news reports, and weather forecasts. A set of low-level audio features are proposed for characterizing semantic contents of short audio clips. The linear separability of different classes under the proposed feature space is examined using a clustering analysis. The effective features are identified by evaluating the intracluster and intercluster scattering matrices of the feature space. Using these features, a neural net classifier wa...

• Type: Journal
• Year: 1998
• Where: VLSISP
• Authors: Zhu Liu, Yao Wang, Tsuhan Chen
• Keywords: Feature Space, Semantic Content, TV Programs, VLSISP 1998