In this paper we propose a robust text segmentation method in complex background. The proposed method first utilizes the K-means algorithm to decompose a detected text block into different binary image layers. Then an effective post-processing is followed to eliminate background residues in each layer. In this step we develop a group of robust constraints to characterize general text regions based on color, edge and stroke thickness. We also propose the components relation constraint (CRC) designed specifically for Chinese characters. Finally the text image layer is identified based on the periodical and symmetrical layout of text lines. The experimental results show that our method can effectively eliminate a wide range of background residues, and has a better performance than the K-means method, as well as a high speed.