We investigate the use of hierarchical Gaussian shortlists to speed up Gaussian likelihood computation. This approach is a combination of hierarchical Gaussian selection and standard Gaussian shortlists. First, all the Gaussians are clustered hierarchically. Then, for the Gaussians in each level of the hierarchy, shortlists are trained to reduce likelihood computation at the corresponding level. This approach enables a hierarchical coarse-to-fine control of the Gaussian likelihood computation. The proposed approach is evaluated in computing the high-dimensional posteriors for feature space Minimum Phone Error (fMPE) front end and also in Viterbi search. Experimental results show that the performance of the proposed approach is superior to using only hierarchical Gaussian selection or standard Gaussian shortlists.