Abstract - A robust method for symbol recognition is presented that utilizes a compact signature based on a modified Hough Transform (HT) and knowledge-based hierarchical neural network structure. Relative position and orientation information is extracted from a symbol image using a modified Hough Transform (HT). This information is transformed and compressed into a compact, 1-D signature vector that is invariant to geometric transformations such as translation, rotation, scaling, and reflection. The proposed method uses a knowledge-based hierarchical neural network structure to reduce the complexity of the recognition process by effectively segmenting the search space into smaller and more manageable clusters based on a priori knowledge. The method achieved overall recognition rates of 96.7% on line graphic symbols from the GREC'05 symbol database under various models of image degradation and distortion.