We present a model of creating a hierarchical set of rules that encode generalizations and exceptions derived from induction learning. The rules use the input features directly and are therefore comprehensible to the users. Learning is performed by a feedforward neural network and the rules are extracted from the trained network. A pattern classification task is used to demonstrate the efficacy of our approach. We show that the rules have similar classification performance while being more comprehensible to the users.