Many traditional relevance feedback approaches for CBIR can only achieve limited short-term performance improvement without benefiting long-term performance. To remedy this limitation, we propose a graphic-theoretic model for incremental relevance feedback in image retrieval. Firstly, a two-layered graph model is introduced that describes the correlations between images. A learning strategy is then suggested to enrich the graph model with semantic correlations between images derived from user feedbacks. Based on the graph model, we propose link analysis approach for image retrieval and relevance feedback. Experiments conducted on real-world images have demonstrated the advantage of our approach over traditional approaches in both short-term and long-term performance.