Given a sequence of observable features of a linear dynamical system (LDS), we propose the problem of finding a representation of the LDS which is sparse in terms of a given dictionary of LDSs. Since LDSs do not belong to Euclidean space, traditional sparse coding techniques do not apply. We propose a probabilistic framework and an efficient MAP algorithm to learn this sparse code. Since dynamic textures (DTs) can be modeled as LDSs, we validate our framework and algorithm by applying them to the problems of DT representation and DT recognition. In the case of occlusion, we show that this sparse coding scheme outperforms conventional DT recognition methods.