The reuse of past coefficient vectors of the NLMS for reducing the steady-state MSD in a low signal-to-noise ratio (SNR) was proposed recently. Its convergence analysis has not been studied yet, so we first derive a steady-state analysis for the NLMS with reusing coefficient vectors for a special case. In addition, this approach slows down the convergence speed while decreasing the steady-state MSD in proportion to the number of reusing coefficient vectors. To address this trade-off, we propose a novel NLMS algorithm which can change the reusing order to achieve both fast convergence speed and low steady-state MSD. The reusing order is decreased or increased by comparing the squared output error with a threshold. The experimental results show that the theoretical results match well with simulation results and the proposed algorithm has fast convergence speed and small steady-state MSD compared to the conventional NLMS.