—In the field of human robot interaction (HRI), providing robot with emotions and psychology like human can be useful to achieve natural interaction. Previous HRI research focused on issues such as gesture recognition, speech recognition and path planning etc. Using the affective information obtained from multimodal interaction can improve the robot’s perception to interaction object and environment, and then can improve the harmoniousness of HRI. The paper puts forward a method of decision-making on multimodal interaction information fusion of service robot based on artificial psychology. The experimental results demonstrate that the proposed method can be efficient and effective in HRI.