With the spread of digital cameras, shooting photos has been becoming an everyday affair. However, there are few methods or systems to manage photos simply, and a huge amount of photo data remains unorganized. Although it is possible to add appropriate words explaining the contents of the photo as one of the methods to manage photos, it requires much time and effort to input such indexes manually. It is also difficult to add indexes intended by a user automatically. This paper proposes a semi-automatic photo indexing system that enables users to generate indexes simply and browse a photo library efficiently. Index candidates are acquired from map database retrieval and relevant word extraction using web retrieval based on shooting position and orientation information. We have implemented an indexing system prototype based on the proposed method, and have carried out some experiments.