RVRL (Rule Value Reinforcement Learning) is a new algorithm which extends an existing learning framework that models the environment of a situated agent using a probabilistic rule representation. The algorithm attaches values to learned rules by adapting reinforcement learning. Structure captured by the rules is used to form a policy. The resulting rule values represent the utility of taking an action if the rule's conditions are present in the agent's current percept. Advantages of the new framework are demonstrated, through examples in a predator-prey environment. Categories and Subject Descriptors I.2.1 [AI/GEN] General Terms Algorithms Keywords Reinforcement learning, perception, action, planning, situated agents, stochastic, environment, probabilistic logic.