We study hierarchical clustering schemes under an axiomatic view. We show that within this framework, one can prove a theorem analogous to one of J. Kleinberg (Kleinberg, 2002), in which one obtains an existence and uniqueness theorem instead of a non-existence result. We explore further properties of this unique scheme: stability and convergence are established. We represent dendrograms as ultrametric spaces and use tools from metric geometry, namely the Gromov-Hausdorff distance, to quantify the degree to which perturbations in the input metric space affect the result of hierarchical methods.