A Formal Proof of a Paradox Associated with Cohen's Kappa

Abstract: Suppose two judges each classify a group of objects into one of several nominal categories. It has been observed in the literature that, for fixed observed agreement between the judges, Cohen's kappa penalizes judges with similar marginals compared to judges who produce different marginals. This paper presents a formal proof of this phenomenon.

• Type: Journal
• Year: 2010
• Where: CLASSIFICATION
• Authors: Matthijs J. Warrens
• Keywords: Automated Reasoning, CLASSIFICATION 2010, Kappa Penalizes Judges, Nominal Categories, Similar Marginals