Dialogue Segmentation with Large Numbers of Volunteer Internet Annotators

This paper shows the results of an experiment in dialogue segmentation. In this experiment, segmentation was done on a level of analysis similar to adjacency pairs. The method of annotation was somewhat novel: volunteers were invited to participate over the Web, and their responses were aggregated using a simple voting method. Though volunteers received a minimum of training, the aggregated responses of the group showed very high agreement with expert opinion. The group, as a unit, performed at the top of the list of annotators, and in many cases performed as well as or better than the best annotator.

• Type: Journal
• Year: 2009
• Where: ACL
• Authors: T. Daniel Midgley
• Keywords: ACL 2009, Computational Linguistics, Dialogue Segmentation, Expert Opinion, Simple Voting Method