This paper presents an infrastructure for developing problem-based pervasive learning environments. Building such environments necessitates having many autonomous components dealing with various tasks and heterogeneous distributed resources. Our proposed infrastructure is based on a multi-agent system architecture to integrate various components of the environments. The infrastructure includes a location- and context-awareness service, a question and answer service, an adaptive mechanism; problem based ubiquitous learning models, social networking issues, and the evaluation of multimedia inputs. Furthermore, student modeling issues among components are considered. The design of the infrastructure as well as its components is discussed. This paves the way towards the development of pervasive learning applications.