Cross-language linking of news stories on the web using interlingual topic modelling

We have studied the problem of linking event information across different languages without the use of translation systems or dictionaries. The linking is based on interlingua information obtained through probabilistic topic models trained on comparable corpora written in two languages (in our case English and Dutch). The achieve this, we expand the Latent Dirichlet Allocation model to process documents in two languages. We demonstrate the validity of the learned interlingual topics in a document clustering task, where the evaluation is performed on Google News. Categories and Subject Descriptors G.3 [Probability and Statistics]: Stochastic Processes; I.2.7 [Artificial Intelligence]: Natural Language Processing—Machine translation; H.3.3 [Information Storage and Retrieval]: Information Search and Retrieval—Clustering General Terms Algorithms, Measurement Keywords Latent Dirichlet Allocation, Event Detection

• Type: Conference
• Year: 2009
• Where: CIKM
• Authors: Wim De Smet, Marie-Francine Moens
• Keywords: CIKM 2009, Database, Dirichlet Allocation Model, Keywords Latent Dirichlet, Latent Dirichlet Allocation