The telling and understanding of stories is a universal part of human experience. If we could reproduce even part of the process inside a computer, it could expand the possibilities for human-computer interaction enormously. We argue that in order to do so, we need to model narrative at three levels of ion, in terms of physics, characters and plot. Taking four scenes from the children’s story The Tale of Peter Rabbit, we describe some of the challenges they present for modeling this kind of “story-sense reasoning”.