For natural images, there are usually repeating similar contents but hard to be well predicted locally. Prediction using template matching is an effective technology to exploit such a non-local correlation. In this paper, we propose an alternative scheme to further exploit the nonlocal correlation. In the proposed scheme, template matching is also used to search for probable similar references to the current block to be coded. We then use these references to train an adaptive transform, which most likely reflects the statistical characteristic of the current block. The proposed scheme can further exploit correlation between the current block and more possible references. Compared to the scheme that only integrates prediction by template matching, the proposed scheme shows improvement about 0.45dB PSNR increase or 9.3% bit saving on average, which leads to 1dB's gain or 19.5% bit saving on average compared to the state-of-the-art scheme without using template matching.