Skin detection or segmentation is employed in many tasks related to the detection and tracking of humans and human-body parts. However, skin detection is not robust enough for dealing with some real-world conditions, such as changing lighting conditions and complex backgrounds containing surfaces and objects with skin-like colors. This situation can be improved by incorporating context information in the skin detection process. For this reason in this article a skin detection approach that uses neighborhood information is proposed. This idea is implemented through a diffusion process that allows a robust segmentation of skin regions at a high processing speed.