We present a framework for media interpretation that leverages low-level information on to a higher level of abstraction in order to support semantics-based information retrieval for the Semantic Web. The overall goal of the framework is to provide high-level content descriptions of documents for maximizing precision and recall of semantics-based information retrieval.1