Evaluating Statistical Tests for Within-Network Classifiers of Relational Data

Recently a number of modeling techniques have been developed for data mining and machine learning in relational and network domains where the instances are not independent and identically distributed (i.i.d.). These methods specifically exploit the statistical dependencies among instances in order to improve classification accuracy. However, there has been little focus on how these same dependencies affect our ability to draw accurate conclusions about the performance of the models. More specifically, the complex link structure and attribute dependencies in network data violate the assumptions of many conventional statistical tests and make it difficult to use these tests to assess the models in an unbiased manner. In this work, we examine the task of within-network classification and the question of whether two algorithms will learn models which will result in significantly different levels of performance. We show that the commonly-used form of evaluation (paired t-test on overlappin...

• Type: Journal
• Year: 2009
• Where: ICDM
• Authors: Jennifer Neville, Brian Gallagher, Tina Eliassi-Rad
• Keywords: Complex Link Structure, Conventional Statistical Tests, Data Mining, ICDM 2009, Type II Error