In this paper a novel framework for three-dimensional surface reconstruction by self-consistent fusion of shading and shadow features is presented. Based on the analysis of at least two pixel-synchronous images of the scene under different illumination conditions, this framework combines a shape from shading approach for estimating surface gradients and altitude variations with a shadow analysis that allows for an accurate determination of altitude differences on the surface. As a first step, the result of shadow analysis is used for selecting a consistent solution of the shape from shading reconstruction algorithm. As a second step, an additional error term derived from the fine-structure of the shadow is incorporated into the reconstruction algorithm. This framework is applied to three-dimensional reconstruction of regions on the lunar surface using groundbased CCD images. Beyond the planetary science scenario, it is applicable to classical machine vision tasks such as surface inspe...