Several domain specific approaches for sports video management have shown the benefits of integrating low- and high- level video contents in supporting more robust retrieval. However, there are very little work has shown how to integrate them in order to support different types of sports. In this paper, we firstly propose a framework for customizable video management system which allows the system to detect the type of video to be indexed, so that appropriate tools can be used to extract the key segments. It is also customizable because the system manages user preferences and usage history to make the system supports specific requirements. Secondly, we will show how the extracted key segments can be summarised using standard descriptions of MPEG-7 in a hierarchical scheme which is potentially easy to share between users. Thirdly, we have developed and tested some queries which show that XQuery provides a powerful language for our video management's retrieval.