Semi-Markov Models for Sequence Segmentation

In this paper, we study the problem of automatically segmenting written text into paragraphs. This is inherently a sequence labeling problem, however, previous approaches ignore this dependency. We propose a novel approach for automatic paragraph segmentation, namely training Semi-Markov models discriminatively using a Max-Margin method. This method allows us to model the sequential nature of the problem and to incorporate features of a whole paragraph, such as paragraph coherence which cannot be used in previous models. Experimental evaluation on four text corpora shows improvement over the previous state-of-the art method on this task.

• Type: Conference
• Year: 2007
• Where: EMNLP
• Authors: Qinfeng Shi, Yasemin Altun, Alex J. Smola, S. V. N. Vishwanathan
• Keywords: Automatic Paragraph Segmentation, EMNLP 2007, Natural Language Processing, Paragraph, Sequence Labeling Problem