Topic Detection by Clustering Keywords

We consider topic detection without any prior knowledge of category structure or possible categories. Keywords are extracted and clustered based on different similarity measures using the induced k-bisecting clustering algorithm. Evaluation on Wikipedia articles shows that clusters of keywords correlate strongly with the Wikipedia categories of the articles. In addition, we find that a distance measure based on the Jensen-Shannon divergence of probability distributions outperforms the cosine similarity. In particular, a newly proposed term distribution taking co-occurrence of terms into account gives best results.

• Type: Conference
• Year: 2008
• Where: DEXAW
• Authors: Christian Wartena, Rogier Brussee
• Keywords: Cosine Similarity, Database, DEXAW 2008, Distribution Taking Co-occurrence, K-bisecting Clustering Algorithm