We introduce a new approach for Clustering and Aggregating Relational Data (CARD). We assume that data is available in a relational form, where we only have information about the degrees to which pairs of objects in the data set are related. Moreover, we assume that the relational information is represented by multiple dissimilarity matrices. These matrices could have been generated using different sensors, features, or mappings. CARD is designed to aggregate pairwise distances from multiple relational matrices, partition the data into clusters, and learn a relevance weight for each matrix in each cluster simultaneously. We introduce two versions of CARD. The first one is completely unsupervised(UCARD). The second version is semi-supervised(SS-CARD) and uses partial supervision information that consists of a small set of must-link and cannot-link constraints. The performance of the proposed algorithms is illustrated by using it to categorize a collection of 500 color images. We repr...