One of disadvantages of Hidden Markov Models (HMMs) is its low resistance to unexpected noises among observation sequences. Unexpected noises in a sequence usually "break" a sequence of observations, and then makes this sequence unrecognizable for trained models. We propose a new HMM training and testing scheme, which compensates some of the negative effects of such noises. We carried out experiment on handwritten digit recognition problem and the result suggests our proposal can be as effective as multiclassifier systems.