The alignment of text line images with text transcript is a crucial step of handwritten document annotation. Handwritten text alignment is prone to errors due to the difficulty of character segmentation and the variability of character shape, size and position. In this paper, we propose to incorporate the geometric context of character strings to improve the alignment accuracy for offline handwritten Chinese documents. We use four statistical models to evaluate the geometric features of single characters and between-character relationships. By combining the geometric models with a character recognizer, we have achieved a large improvement of alignment accuracy in our experiments on unconstrained handwritten Chinese text lines.