In this paper a new approach to finding and tracking various land cover boundaries such as rivers, agricultural fields, channels and roads for use in visual navigation system of an unmanned aerial vehicle is presented. We use a combination of statistical estimation and optimization techniques for extraction of dominant boundaries in noisy aerial images. A set of perceptual grouping restrictions is used to connect the acquired piecewise boundaries and to find the heading direction of the main boundary. The results are further refined by applying a set of texture and colour cues and eliminating any false hypothesis. Our results show our method outperforms single feature object tracking algorithms in this application.