This paper describes a method of generating a Kanji hierarchical structured dictionary for stroke-number and stroke-order free handwriting recognition based on substroke HMM. In stroke-based methods, a large number of stroke-order variations can be easily expressed by just adding different stroke sequences to the dictionary and it is not necessary to train new reference patterns. The hierarchical structured dictionary has an advantage that thousands of stroke-order variations of Kanji characters can be produced using a small number of stroke-order rules defining Kanji parts. Moreover, the recognition speed is fast since common sequences are shared in a substroke network, even if the total number of stroke-order combinations becomes enormous practically. In experiments, 300 different stroke-order rules of Kanji parts were statistically chosen by using 60 writers’ handwritings of 1,016 educational Kanji characters. By adding these new stroke-order rules to the dictionary, about 9,0...