The success of RFID in supply chain management is leading many to consider more personal and pervasive deployments of this technology. Unlike industrial settings, however, deployments that involve humans raise new and critical problems related to privacy, security, uncertainty, and a more diverse and evolving set of applications. At the University of Washington, we are deploying a building-wide RFID-based infrastructure with hundreds of antennas and thousands of tags. Our goal is to uncover the issues of pervasive RFID deployments and devise techniques for addressing these issues before such deployments become common place. In this paper, we present the challenges encountered and lessons learned during a smaller-scale pilot deployment of the system. We show some preliminary results and, for each challenge, discuss how we addressed it or how we are planning on addressing it.