Unsupervised Learning of Dependency Structure for Language Modeling

This paper presents a dependency language model (DLM) that captures linguistic constraints via a dependency structure, i.e., a set of probabilistic dependencies that express the relations between headwords of each phrase in a sentence by an acyclic, planar, undirected graph. Our contributions are three-fold. First, we incorporate the dependency structure into an n-gram language model to capture long distance word dependency. Second, we present an unsupervised learning method that discovers the dependency structure of a sentence using a bootstrapping procedure. Finally, we evaluate the proposed models on a realistic application (Japanese Kana-Kanji conversion). Experiments show that the best DLM

• Type: Conference
• Year: 2003
• Where: ACL
• Authors: Jianfeng Gao, Hisami Suzuki
• Keywords: ACL 2003, ACL 2007, Dependency Language Model, Dependency Structure, Distance Word Dependency