For the tasks of classification, two types of patterns can generate problems: ambiguous patterns and outliers. Furthermore, it is possible to separate classification algorithms into two main categories. Discriminative approaches try to find the better separation among all classes and minimize the first type of error. But, in general they cannot deal with outliers. Besides, modelbased approaches make the outlier detection possible but are not sufficiently discriminative. Thus, we propose to combine a model-based approach with support vectors classifiers (SVC) in a two-stage classification system. Another advantage of this combination is to reduce the principal burden of SVC: the processing time necessary to make a decision. Finally, the experiments on handwriting digit recognition have shown that it is possible to maintain the accuracy of SVCs, while decreasing complexity significantly.