The work presented in this paper aims at reducing the semantic gap between low level video features and semantic video objects. The proposed method for finding associations between segmented frame region characteristics relies on the strength of Latent Semantic Analysis. Our previous experiments [1] have shown the potential of this approach but also uncovered some of its limitation. Here, we will present a method using the structural information within an LSA framework. Moreover, we will demonstrate the performance gain of combining visual (low level) and structural information.