This paper shows an approach for a storytelling oriented interaction on digital video. All the interaction capabilities of the system are driven by the video context and therefore media centric. Interaction possibilities are given to the audience by conversation (on topics of the video content) or by classical Direct Manipulation (of video objects). Conversations can be done by the user with a personalized assistance or directly with the video. The implementation of the approach is shown by a discussion about an application architecture on the basis of Real Media Server, SMIL and Java 3D. The application runs as a Video On Demand system, accessible through the world wide web.