In recent years, adaptation techniques have been given special focus in speaker recognition tasks, mainly targeting speaker and session variation disentangling under the Maximum a Posteriori (MAP) criterion. For these techniques, unseen mixtures are usually adapted in a global manner, if ever. In this paper, we explore Structural MAP (SMAP), Maximum a Posteriori adaptation using hierarchical structures of the acoustic space that allow data scarceness issues to be tackled with different precision levels. We explore this approach in a speaker verification system using a Support Vector Machine (SVM) classifier and Gaussian mean supervectors (GMM-SVM). We show that this is an effective approach that considerably outperforms its relevance MAP counterpart in the 2006 NIST Speaker Recognition Evaluation. We also show that using a speaker-adapted Universal Background Model can improve the stability of the clustering algorithm besides obtaining further improvements.