Merging Algorithm to Reduce Dimensionality in Application to Web-Mining

Dimensional reduction may be effective in order to compress data without loss of essential information. Also, it may be useful in order to smooth data and reduce random noise. The model presented in this paper was motivated by the structure of the msweb web-traffic dataset from the UCI archive. It is proposed to reduce dimension (number of the used web-areas or vroots) as a result of the unsupervised learning process maximizing a specially defined average log-likelihood divergence. Two different web-areas will be merged in the case if these areas appear together frequently during the same sessions. Essentially, roles of the web-areas are not symmetrical in the merging process. The web-area or cluster with bigger weight will act as an attractor and will stimulate merging. In difference, the smaller cluster will try to keep independence. In both cases the powers of attraction or resistance will depend on the weights of the corresponding clusters. The above strategy will prevent creat...

• Type: Conference
• Year: 2007
• Where: AUSAI
• Authors: Vladimir Nikulin, Geoffrey J. McLachlan
• Keywords: Artificial Intelligence, AUSAI 2007, Average Log-likelihood Divergence, Msweb Web-traffic Dataset, Unsupervised Learning Process