The object boundary of an image plays an important role for image analysis and interpretation. The watershed algorithm and the region growing algorithm are popularly employed for image segmentation. These give reasonable performances, but require a large amount of computation time and sometimes fail to obtain continuous linkage of object boundary. In this paper, we introduce hierarchical mesh-based image segmentation. In each hierarchy, we employ neighborhood searching and boundary tracking methods to refine the initial boundary estimate. The proposed algorithm increases the robustness of linkage of object boundaries by overlooking and estimating connectivity and gives new modified chain coding. Reliable segmentation of objects can be accomplished by the proposed low complexity technique.