Plan recognition has traditionally been developed for logically encoded application domains with a focus on logical reasoning. In this paper, we present an integrated plan-recognition model that combines low-level sensory readings with high-level goal inference. A twolevel architecture is proposed to infer a user's goals in a complex indoor environment using an RF-based wireless network. The novelty of our work derives from our ability to infer a user's goals from sequences of signal trajectory, and the ability for us to make a tradeoff between model accuracy and inference efficiency. The model relies on a dynamic Bayesian network to infer a user's actions from raw signals, and an N-gram model to infer the users' goals from actions. We present a method for constructing the model from the past data and demonstrate the effectiveness of our proposed solution through empirical studies using some real data that we have collected.