In this paper we summarize the experiences we made with the implementation of a table tennis application. After describing the hardware necessities of our system we give insight into different aspects of the simulation. These include collision detection, physical simulation and some aspects of the design of the virtual opponent. Since table tennis is one of the fastest sports the synchronization of the player’s movements and the visual output on the projection wall is the most challenging problem to solve. Therefore we analysed the latencies of all subcomponents of our system and designed a prediction method that allows high speed interaction with our application. Categories and Subject Descriptors B.8.2 [Performance and Reliability]: Performance Analysis and Design Aids. General Terms Algorithms, Measurement, Performance, Design Keywords Virtual Reality, Tracking, Latency, Prediction, Collision Detection, Table tennis