Nowadays, video games propose rich scenarios with movementsbased game play, through the manipulation of dedicated devices. The motivation of this study is the elaboration of a complete interactive system upon which scenario-based applications operate. The user interacts with an immersive 3D environment by his body movements, captured by a non invasive interface. The system interprets the user behaviors thanks to the support of the application scenario. This interpretation drives the system responses. The management of the context of a given scenario situation has allowed the optimization of the whole system. The contextual information not only helps to determine the true meaning of an observed behavior but also makes the system to adapt its processes regarding this interpretation, while managing its hardware and software resources efficiently. A commercial motion capture interface has been enhanced by the elaboration of such a system. A precise framework to develop applications is bei...