For virtual reality systems, modeling of 3D objects and scenes is important and challenging. In this paper, we present an image-based interactive 3D modeling framework consisting of three major modules: photogrammetric modeling, human interaction, and texture mapping. These three modules are not sequentially used and they are mixed in the whole modeling process. The major idea is to explore the use of images in interactive modeling systems to achieve the automation. In particular, the use of only one image is addressed. On one side, unlike the common fully interactive modeling framework, the users are not required to specify some low level details interactively which can be derived automatically from the image. On the other side, it still requires human interactions to do some high level tasks that the algorithms are difficult to perform automatically. We have implemented the framework and experimental results are good. Categories and Subject Descriptors I.3.5 [Computer Graphics]: Com...