This paper presents a method for visual object categorization based on encoding the joint textural information in objects and the surrounding background, and requiring no segmentation during recognition. The framework can be used together with various learning techniques and model representations. Here we use this framework with simple probabilistic models and more complex representations obtained using Support Vector Machines. We prove that our approach provides good recognition performance for complex problems for which some of the existing methods have difficulties. Additionally, we introduce a new extensive database containing realistic images of animals in complex natural environments. We asses the database in a set of experiments in which we compare the performance of our approach with a recently proposed method.