In this paper we derive formal constraints relating terrain elevation and observed cast shadows. We show how an optimisation framework can be used to refine surface estimates using shadowing constraints from one or more images. The method is particularly applicable to the digital elevation models produced by the Shuttle Radar Topography Mission (SRTM), which have an abundance of voids in mountainous areas where elevation data is missing. Cast shadow maps are detected automatically from multispectral satellite imagery using a simple heuristic which is reliable over varying types of surface cover. We show that the combination of our shadow segmentation and terrain correction methods can restore the structure of mountain ridges in interpolated SRTM voids using five satellite images, decreasing the RMS error by over 25%.