Abstract. It is difficult to understand a scene from visual information in uncertain real world. Since Bayesian network (BN) is known as good in this uncertainty, it has received significant attention in the area of vision-based scene understanding. However, BN-based modeling methods still have the difficulties in modeling complex relationships and combining several modules, as well as the high computational complexity of inference. To overcome them, this paper proposes a method to divide and select the BN modules for recognizing the objects in uncertain environments. The method utilizes the behavior selection network to select the most appropriate BN modules. Several experiments are performed to verify the usefulness of the proposed method.