This paper proposes an efficient skin-color and template based technique for automatic ear detection in a side face image. The technique first separates skin regions from nonskin regions and then searches for the ear within skin regions. Ear detection process involves three major steps. First, Skin Segmentation to eliminate all non-skin pixels from the image, second Ear Localization to perform ear detection using template matching approach, and third Ear Verification to validate the ear detection using the Zernike moments based shape descriptor. To handle the detection of ears of various shapes and sizes, an ear template is created considering the ears of various shapes (triangular, round, oval and rectangular) and resized automatically to a size suitable for the detection. Proposed technique is tested on the IIT Kanpur ear database consisting of 150 side face images and gives 94% accuracy.