Spherical harmonics are widely used in 3D image processing due to their compactness and rotation properties. For example, it is quite easy to obtain rotation invariance by taking the magnitudes of the representation, similar to the power spectrum known from Fourier analysis. We propose a novel approach extending the spherical harmonic representation to tensors of higher order in a very efficient manner. Our approach utilises the so called tensorial harmonics [1] to overcome the restrictions to scalar fields. In this way it is possible to represent vector and tensor fields with all the gentle properties known from spherical harmonic theory. In our experiments we have tested our system by using the most commonly used tensors in three dimensional image analysis, namely the gradient vector, the Hessian matrix and finally the structure tensor. For comparable results we have used the Princeton Shape Benchmark [2] and a database of airborne pollen, leading to very promising results.