The first contribution of this paper is a probabilistic approach for measuring motion similarity for point sequences. While most motion segmentation algorithms are based on a rank-constraint on the space of affine motions, our method is based on spectral clustering of a probability measure for motion similarity which can be applied to any parametric model. The probabilistic framework allows for incorporation of informative priors for the noise and camera motion. Similarly spatial and temporal priors can also be subsumed leading to useful segmentation techniques. Our second contribution is a tensor-decomposition technique enables us to infer motion affinity from higher dimensional representations. Results are presented on real image sequences.