This paper presents a novel model of reinforcement learning agents. A feature of our learning agent model is to integrate analytic hierarchy process (AHP) into a standard reinforcement learning agent model, which consists of three modules: state recognition, learning, and action selecting modules. In our model, AHP module is designed with primary knowledge that human intrinsically should have in order to attain a goal state. This aims at increasing promising actions of agent especially in the earlier stages of learning instead of completely random actions as in the standard reinforcement learning algorithms. We adopt profit-sharing as a reinforcement learning algorithm and demonstrate the potential of our approach on two learning problems of a pursuit problem and a Sokoban problem with deadlock in the grid-world domains, where results indicate that the learning time can be decreased considerably for the problems and our approach efficiently avoids the deadlock for the Sokoban problem...