This paper presents a framework for building rule-based image retrieval (RBIR) systems. Soft computing based multimedia data mining techniques are employed to extract and optimize fuzzy rules that infer semantic concepts for image retrieval. Using fuzzy inference system (FIS), a novel RBIR system with semantic query is developed. Experimental results on an artificial image databse are reported.