This article describes the Xtrieval framework - a flexible platform for testing and evaluating different aspects of retrieval systems. The background for the design of the framework is to develop an extensible platform for the combination of text and content-based multimedia retrieval. We describe the general composition of the framework and its main functionality. An evaluation of the framework capabilities is given with two exemplary retrieval evaluation tasks.