In this paper, we propose a new method to cross-media semantic-based information retrieval, which combines classical text-based and content-based image retrieval techniques. This semanticbased approach aims at determining the strong relationships between keywords (in the caption) and types of visual features associated with its typical images. These relationships are then used to retrieve images from a textual query. In particular, the association keyword/visual feature/characterization may allow us to retrieve non-annotated but similar images to those retrieved by a classical textual query. It can also be used for automatic images annotation. Our experiments on two different databases show that this approach is promising for cross-media retrieval.