The basic idea of Lucas and Kanade is to constrain the local motion measurement by assuming a constant velocity within a spatial neighborhood. We reformulate this spatial constraint in a probabilistic way assuming Gaussian distributed uncertainty in spatial identification of velocity measurements and extend this idea to scale and time dimensions. Thus, we are able to combine uncertain velocity measurements observed at different image scales and positions over time. We arrive at a new recurrent optical flow filter formulated in a Dynamic Bayesian Network applying suitable factorisation assumptions and approximate infer