Audio and video signatures for synchronization

Abstract— We propose a framework based on signatures extracted from audio and video streams for automatically measuring and maintaining synchronization between the two streams. The audio signature is based on projections of a coarse representation of the spectrogram onto random vectors. The video signature is based on projections of a coarse representation of the difference image between two consecutive frames onto random vectors. The time alignment present at the signature generator between the two streams is recorded by combining audio and video signatures into a combined synchronization signature. At the detector after video and audio streams go through different processing operations, we extract the signatures again. The signatures extracted before and after processing from the audio and the video are compared independently using a Hamming distance based correlator to estimate the relative misalignment introduced due to processing in each of the streams. Then, the estimated relat...

• Type: Conference
• Year: 2008
• Where: ICMCS
• Authors: Regunathan Radhakrishnan, Kent Terry, Claus Bauer
• Keywords: ICMCS 2008, Multimedia, Signatures, Video Signatures, Video Streams