We propose a method to calibrate a rotating and zooming camera without 3D pattern, where the internal parameters change frame by frame. First, we show that the calibration is unique up to an orthogonal transformation under the assumption that the skew of the camera is zero. The auto-calibration is possible by analyzing inter-image homographies computed from the matches in the images of the same scene. At least four homographies are needed for auto-calibration in general. When we assume that the aspect ratio is known and the principal point is fixed then one homography will yield camera parameters, and when the aspect ratio is not known with fixed principal point then two homographies are enough. The algorithm is implemented and validated on several sets of synthetic data and real image data.