Ontology learning is an important task in Artificial Intelligence, Semantic Web and Text Mining. This paper presents a novel framework for, and solutions to, three practical problems in ontology learning. An incremental clustering approach is used to solve the problem of unknown group names. Learned models at each level of an ontology address the of no control over concept abstractness. A metric learning module moves beyond the limitation of traditional use of features and incorporates heterogeneous semantic evidence into the learning process. The metric-based learning framework integrates these separate components into a single, unified solution. An extensive evaluation with WordNet and Open Directory Project data demonstrates that the method is more effective than a state-of-the-art baseline algorithm. Categories and Subject Descriptors H.4 [Information Systems Applications]: Miscellaneous Keywords Learning, Concept Abstractness, Ontology Metric