We propose a hierarchical disparity estimation algorithm with energy-based regularization. Initial disparity vectors are obtained from downsampled stereo images using a feature-based region-dividing disparity estimation technique. Dense disparities are estimated from these initial vectors with shape-adaptive windows in full resolution images. Finally, the vector fields are regularized with the minimization of the energy functional which considers both fidelity and smoothness of the fields. The first two steps provide highly reliable disparity vectors, so that local minimum problem can be avoided in regularization step. The proposed algorithm generates accurate disparity map which is smooth inside objects while preserving its discontinuities in boundaries. Experimental results are presented to illustrate the capabilities of the proposed disparity estimation technique.