Traditional channel quantization based methods for encryption key generation usually suffer from the quantization error which may decrease the key agreement ratio between authorized users. In this paper, we make use of the reciprocity of the wireless channel in time-division duplex (TDD) mode, and adopt a closed-form robust Chinese remainder theorem (CRT) to extract the encryption key from the noisy channel estimations. The channel coefficients are regarded as the remainders in robust CRT, and the key is generated from the quotient output of the robust CRT. In addition, a remainder selection procedure is proposed to further improve its performance. Simulations demonstrate the effectiveness of the proposed method.