This paper presents a semi-automated segmentation method for short-axis cardiac CT and MR images. The main contributions of this work are: 1) using two different energy functionals for endocardium and epicardium segmentation to account for their distinctive characteristics; 2) proposing a dual-background model that is suitable for representing intensity distributions of the background in epicardium segmentation; 3) designing a novel shape prior term that is robust and controllable; and 4) an improved estimation of myocardium thickness by using edge information. Experimental results on cardiac CT, perfusion and cine MR images show that our method is robust and effective for both CT and MR images.