It is very common that a user likes to collect many multimedia files of their interests from the web or other sources for his/her daily use, such as in emails, presentations, and technical documents. This paper presents algorithms to learn user models, in particular, user intention models and preference models from the usage of these files. Such usages include downloading, inserting, and sending multimedia files. A user intention model predicts when the user may want to involve some multimedia objects in his currently working environment (e.g., an email) and provides more convenient and accurate help to the user. A user preference model describes the types and classes of the user’s favorite multimedia files and helps an offline crawler to autonomously collect more useful multimedia files for the user. The algorithms have been implemented in our media agents system and shown their effectiveness in user modeling.