In this paper, we propose an algorithm that recovers binocular disparities in accordance with the surface properties of the scene under consideration. To do so, we estimate the disparity as well as the normals in the disparity space, by setting the two tasks in a unified framework. A novel joint probabilistic model is defined through two random fields to favor both intra field (within neighboring disparities and neighboring normals) and inter field (between disparities and normals) consistency. Geometric contextual information is introduced in the models for both normals and disparities, which is optimized using an appropriate alternating maximization procedure. We illustrate the performance of our approach on synthetic and real data.