Voting experts: An unsupervised algorithm for segmenting sequences

We describe a statistical signature of chunks and an algorithm for finding chunks. While there is no formal definition of chunks, they may be reliably identified as configurations with low internal entropy or unpredictability and high entropy at their boundaries. We show that the log frequency of a chunk is a measure of its internal entropy. The Voting-Experts exploits the signature of chunks to find word boundaries in text from four languages and episode boundaries in the activities of a mobile robot.

• Type: Journal
• Year: 2007
• Where: IDA
• Authors: Paul R. Cohen, Niall M. Adams, Brent Heeringa
• Keywords: IDA 2007, Information Technology, Internal Entropy, Low Internal Entropy, Statistical Signature