The paper presents a new vector quantization based approach for selecting well-suited data for hand-eye calibration from a given sequence of hand and eye movements. Data selection is essential if control of the movements used for calibration is not possible, especially when using continuously recorded data. The new algorithm is compared to another method for data selection as well as to the processing of subsequent movements. Experimental results on real and synthetic data sets show the superior performance of the new approach with respect to calibration errors and computation time. Real data has been obtained from an optical tracking system and a camera mounted on an endoscope, the goal being the reconstruction of medical light fields.