The filter flow problem is to compute a space-variant
linear filter that transforms one image into another. This
framework encompasses a broad range of transformations
including stereo, optical flow, lighting changes, blur, and
combinations of these effects. Parametric models such as
affine motion, vignetting, and radial distortion can also be
modeled within the same framework. All such transformations
are modeled by selecting a number of constraints and
objectives on the filter entries from a catalog which we enumerate.
Most of the constraints are linear, leading to globally
optimal solutions (via linear programming) for affine
transformations, depth-from-defocus, and other problems.
Adding a (non-convex) compactness objective enables solutions
for optical flow with illumination changes, spacevariant
defocus, and higher-order smoothness.
Steven M. Seitz, Simon Baker