This paper illustrates how canonical correlation analysis can be used for designing efficient visual operators by learning. The approach is highly task oriented and what constitutes the relevant information is defined by a set of examples. The examples are pairs of images displaying a strong dependence in the chosen feature but are otherwise independent. Experimental results are presented illustrating the learning of local shift invariant orientation operators, representation of velocity, and image content invariant disparity operators.