— In this paper, an approach to interaction control of a robot manipulator with a partially known environment is proposed. The environment is a rigid object of known geometry but of unknown and possibly time-varying position and orientation. An algorithm for online estimation of the object pose is adopted, based on visual data provided by a camera as well as on forces and moments measured during the interaction with the environment. This algorithm is embedded into an impedance control scheme, resulting in a position-based visual impedance control. Experimental results are presented for the case of an industrial robot manipulator in contact with a planar surface.