Statistical appearance models are valuable tools in medical image segmentation. Current methods elegantly incorporate global shape and appearance, but can not cope with local appearance variations and rely on an assumption of Gaussian gray value distribution. Furthermore, initialization near the optimal solution is required. We propose a shape inference method that is based on pixel classification, so that local and non-linear intensity variations are dealt with naturally, while a global shape model ensures a consistent segmentation. Optimization by stochastic sampling removes the need for accurate initialization. The method is demonstrated on vertebra segmentation in spine radiographs. Segmentation errors are below 2 mm in