Statistical shape analysis techniques commonly employed in the medical imaging community, such as Active Shape Models or Active Appearance Models, rely on Principal Component Analysis (PCA) to decompose shape variability into a reduced set of interpretable components. In this paper we propose Principal Factor Analysis (PFA) as an alternative and complementary tool to PCA providing a decomposition into modes of variation that can be more easily interpretable, while still being a linear efficient technique that performs dimensionality reduction (as opposed to Independent Component Analysis, ICA). The key difference between PFA and PCA is that PFA models covariance between variables, rather than the total variance in the data. The added value of PFA is illustrated on 2D landmark data of corpora callosa outlines. Then, a study of the 3D shape variability of the human left femur is performed. Finally, we report results on vector-valued 3D deformation fields resulting from non-rigid registr...