In this paper, we deal with a generative model for multi-label, interactive segmentation. To estimate the pixel likelihoods for each label, we propose a new higher-order formulation additionally imposing the soft label consistency constraint whereby the pixels in the regions, generated by unsupervised image segmentation algorithms, tend to have the same label. In contrast with previous works which focus on the parametric model of the higher-order cliques for adding this soft constraint, we address a nonparametric learning technique to recursively estimate the region likelihoods as higher-order cues from the resulting likelihoods of pixels included in the regions. Therefore the main idea of our algorithm is to design two quadratic cost functions of pixel and region likelihoods, that are supplementary to each other, in a proposed multi-layer graph and to estimate them simultaneously by a simple optimization technique. In this manner, we consider long-range connections between the regions...