In this paper we address the problem of reconstructing a structurally simple surface representation from point datasets of scanned scenes as they occur for instance in city scanning projects. Such datasets generally suffer from all kinds of scanning artifacts including noise, outliers, holes and irregular/anisotropic sampling. Most common surface reconstruction methods fail due to these shortcomings. We present a robust and efficient method to compute local properties for each data point, use this information to detect simple primitive shapes in the data and describe a novel method to extract and optimize boundary curves on the primitive shapes. Employing the reconstructed boundary curves, we are able to extract a piecewise smooth surface mesh which is mandatory for many further processing applications including visualization. We show the effectiveness of our method with reconstructions of synthetic datasets as well as laserscanner-acquisitions of indoor and outdoor environments.