Sciweavers

ICASSP
2010
IEEE

Using the Amazon Mechanical Turk for transcription of spoken language

14 years 23 days ago
Using the Amazon Mechanical Turk for transcription of spoken language
We investigate whether Amazon’s Mechanical Turk (MTurk) service can be used as a reliable method for transcription of spoken language data. Utterances with varying speaker demographics (native and non-native English, male and female) were posted on the MTurk marketplace together with standard transcription guidelines. Transcriptions were compared against transcriptions carefully prepared in-house through conventional (manual) means. We found that transcriptions from MTurk workers were generally quite accurate. Further, when transcripts for the same utterance produced by multiple workers were combined using the ROVER voting scheme, the accuracy of the combined transcript rivaled that observed for conventional transcription methods. We also found that accuracy is not particularly sensitive to payment amount, implying that high quality results can be obtained at a fraction of the cost and turnaround time of conventional methods.
Matthew Marge, Satanjeev Banerjee, Alexander I. Ru
Added 06 Dec 2010
Updated 06 Dec 2010
Type Conference
Year 2010
Where ICASSP
Authors Matthew Marge, Satanjeev Banerjee, Alexander I. Rudnicky
Comments (0)