Skin color is an important feature of faces. Various applications benefit from robust skin color detection. Depending on camera settings, illumination, shadows, people's tans, and ethnic groups skin color looks differently, which is a challenging aspect for detecting it automatically. In this paper, we present an approach that uses a high level vision module to detect an image specific skin color model. This model is then used to adapt parametric skin color classifiers to the processed image. This approach is capable to distinguish skin color from extremely similar colors, such as lip color or eyebrow color. Its high speed and high accuracy make it appropriate for real time applications such as face tracking and recognition of facial expressions.