Energy functional minimization is an increasingly popular technique for image segmentation. However, it is far too commonly applied with hand-tuned parameters and initializations that have only been validated for a few images. Fixing these parameters over a set of images assumes the same parameters are ideal for each image. We highlight the effects of varying the parameters and initialization on segmentation accuracy and propose a framework for attaining improved results using image adaptive parameters and initializations. We provide an analytical definition of optimal weights for functional terms through an examination of segmentation in the context of image manifolds, where nearby images on the manifold require similar parameters and similar initializations. Our results validate that fixed parameters are insufficient in addressing the variability in real clinical data, that similar images require similar parameters, and demonstrate how these parameters correlate with the image manifo...