There has been significant recent interest in sparse metric learning (SML) in which we simultaneously learn both a good distance metric and a low-dimensional representation. Unfortunately, the performance of existing sparse metric learning approaches is usually limited because the authors assumed certain problem relaxations or they target the SML objective indirectly. In this paper, we propose a Generalized Sparse Metric Learning method (GSML). This novel framework offers a unified view for understanding many of the popular sparse metric learning algorithms including the Sparse Metric Learning framework proposed in [15], the Large Margin Nearest Neighbor (LMNN) [21][22], and the D-ranking Vector Machine (D-ranking VM) [14]. Moreover, GSML also establishes a close relationship with the Pairwise Support Vector Machine [20]. Furthermore, the proposed framework is capable of extending many current non-sparse metric learning models such as Relevant Vector Machine (RCA) [4] and a state-of-t...