This paper deals with the problem of finding the optimal sequence of sub-word unit HMMs for a number of given utterances of a word. For this problem we present a new solution based on an extension of the Viterbi algorithm which maximizes the joint probability of the utterances and all possible sequences of sub-word units and hence guarantees to find the optimal solution. The new algorithm was applied in an isolated word recognition experiment and compared to simpler approaches to determining the sequence of sub-word units. We report a significant reduction of the recognition error rate with the new algorithm.