Distributional term representations: an experimental comparison

A number of content management tasks, including term categorization, term clustering, and automated thesaurus generation, view natural language terms (e.g. words, noun phrases) as first-class objects, i.e. as objects endowed with an internal representation which makes them suitable for explicit manipulation by the corresponding algorithms. The information retrieval (IR) literature has traditionally used an extensional (aka distributional) representation for terms according to which a term is represented by the “bag of documents” in which the term occurs. The computational linguistics (CL) literature has independently developed an alternative distributional representation for terms, according to which a term is represented by the “bag of terms” that co-occur with it in some document. This paper aims at discovering which of the two representations is most effective, i.e. brings about higher effectiveness once used in tasks that require terms to be explicitly represented and ma...

• Type: Conference
• Year: 2004
• Where: CIKM
• Authors: Alberto Lavelli, Fabrizio Sebastiani, Roberto Zanoli
• Keywords: CIKM 2004, Natural Language, Term Categorization, Term Clustering