Abstract. We report our work towards building user models of learner’s development based upon evidence of their interactions with an e-learning website composed of multimedia learning objects. Essentially, this involves three elements: effective metadata for each learning object; analysis of each user’s time spent with each learning object; reasoning about each individual’s knowledge of the course that is based upon a collection of learning objects. In this chapter, we focus on the problem of creating the metadata in a manner that will support effective user modelling. The chapter begins with a brief discussion of our overall architecture for building a user model and defining the metadata for multi-media learning objects. We then describe Metasaur, the interface used to create the metadata, and SIV, the visualisation interface to support users in scrutinising the reasoning about them. We also briefly describe the process used to construct an ontology automatically from an existi...