In contextual computing, where cues beyond direct user input are used to trigger computation, one of the most daunting challenges is inferring what the user is doing. For the domain of task management, we have developed a new approach to reducing the problem of ambiguity of user action for intelligent systems. We introduce a construct we call an Activity, designed to reduce this ambiguity by providing a meaningful structure for task information that assists users with their work. We present ethnographic research and prototype evaluations to assess the value of the Activity construct from an end-user’s perspective. Our findings suggest that the Activity structure is useful to people and therefore could be exploited for inference.