This paper deals with the problem of incorporating natural regularity conditions on the motion in an MAP estimator for structure and motion recovery from uncalibrated image sequences. The purpose of incorporating these constraints is to increase performance and robustness. Autocalibration and structure and motion algorithms are known to have problems with (i) the frequently occurring critical camera motions, (ii) local minima in the non-linear optimization and (iii) the high correlation between different intrinsic and extrinsic parameters of the camera, e.g. the coupling between focal length and camera position. The camera motion (both intrinsic and extrinsic parameters) is modelled as a random walk process, where the inter-frame motions are assumed to be independently normally distributed. The proposed scheme is demonstrated on both simulated and real data showing the increased performance.