The recent advanced representation for realistic real-world materials in virtual reality applications is the Bidirectional Texture Function (BTF) which describes rough texture appearance for varying illumination and viewing conditions. Such a function can be represented by thousands of measurements (images) per material sample. The resulting BTF size excludes its direct rendering in graphical applications and some compression of these huge BTF data spaces is obviously inevitable. In this paper we present a novel, fast probabilistic model-based algorithm for realistic BTF modeling allowing an extreme compression with the possibility of a fast hardware implementation. Its ultimate aim is to create a visual impression of the same material without a pixel-wise correspondence to the original measurements. The analytical step of the algorithm starts with a BTF space segmentation and a range map estimation by photometric stereo of the BTF surface, followed by the spectral and spatial factori...