This paper describes a wearable annotation overlay system using augmented reality (AR) techniques. To realize AR systems, the position and orientation of user’s viewpoint should be obtained in real time. The proposed system measures the orientation of user’s viewpoint by an inertial sensor and the user’s position using positioning infrastructures in environments and a pedometer. The system specifies the user’s position using the position ID received from RFID tags or IrDA markers which are the components of positioning infrastructures. When the user goes away from them, the user’s position is alternatively estimated by using a pedometer.