Abstract. In this paper we introduce a principled approach to modeling the image brightness constraint for optical flow algorithms. Using a simple noise model, we derive a probabilistic representation for optical flow. This representation subsumes existing approaches to flow modeling, provides insights into the behaviour and limitations of existing methods and leads to modified algorithms that outperform other approaches that use the brightness constraint. Based on this representation we develop algorithms for flow estimation using different smoothness assumptions, namely constant and affine flow. Experiments on standard data sets demonstrate the superiority of our approach.