Video adaptation has been considered as a promising technique to tackle challenging problems in pervasive multimedia applications. However, the styles of video representation and description in existing framework are not flexible enough to adapt diversified application environment. In this paper, we propose a novel solution based on intermediate description, which can support fast multiple video adaptation operations in signal level (e.g., temporal resolution reduction, bit-rate adaptation), structural level (e.g., random access of any shots, fast preview of any keyframes or thumbnails), as well as joint level. Experimental results show that the presented solution can support the operations in real-time environment, while maintaining coding efficiency, which demonstrates its feasibility and effectiveness.