The role of physical gestures in human-computer interaction has mostly been neglected. In this paper, we argue that gestures are not just one input modality. Rather, they should be seen as a unifying phenomenon, in terms of which all interaction with physical reality could be conceptualised.