Studying the genetic control of molecular, anatomical and/or morphological phenotypes in model organisms is a powerful tool in the functional analysis of a gene. The goal of our research is to develop algorithms that discover phenotypes of behavior in model organisms, which may identify, categorize, and quantify these phenotypes under conditions of minimal a priori information. Starting from a non-invasive video monitoring of a model organism, we propose an eigen-decomposition of the organism’s behavior captured in video. Traditional clustering techniques in space, time, and frequency can utilize this decomposition to characterize the categorical behaviors of an animal, and for an analysis of the behavioral repertoire. This supplies a quantified analysis of behavior with minimal assumptions, a crucial first step in the genetic analysis of behavior.