We present a method to classify materials in illumination series data. An illumination series is acquired using a device which is capable to generate arbitrary lighting environments covering nearly the whole space of the upper hemisphere. The individual images of the illumination series span a high-dimensional feature space. Using a random forest classifier different materials, which vary in appearance (which itself depends on the patterns of incoming illumination), can be distinguished reliably. The associated Gini feature importance allows for determining the features which are most relevant for the classification result. By linking the features to illumination patterns a proposition about optimal lighting for defect detection can be made, which yields valuable information for the selection and placement of light sources.